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Overview 



Young people who drop out of high school face long odds of success in a labor market that in- 
creasingly values education and skills. This report presents interim results from a rigorous, ongoing 
evaluation of the National Guard Youth ChalleNGe Program, which aims to “reclaim the lives of at- 
risk youth” who have dropped out of high school. ChalleNGe is an intensive residential program that 
currently operates in more than half the states. More than 90,000 young people have completed the 
program since it was launched in the early 1990s. MDRC, a nonprofit, nonpartisan research organi- 
zation, is conducting the evaluation in collaboration with the Mac Arthur Foundation Research 
Network on Transitions to Adulthood. Several private foundations and the U.S. Department of 
Defense are funding the evaluation. 

The 17-month ChalleNGe program is divided into three phases: Pre-ChalleNGe, a demanding two- 
week orientation and assessment period; a 20-week Residential Phase built around eight core 
components designed to promote positive youth development; and a one-year Postresidential Phase 
featuring a structured mentoring program. During the first two phases, participants live at the 
program site, often on a military base. The environment is “quasi-military,” though there are no 
requirements for military service. 

The evaluation uses a random assignment design. Because there were more qualified applicants than 
slots, a lottery-like process was used to decide which applicants were admitted to the program. The 
young people who were admitted (the program group) are being compared over time with those who 
were not admitted (the control group); any significant differences that emerge between the groups 
can be attributed to ChalleNGe. About 3,000 young people entered the study in 10 ChalleNGe 
programs in 2005-2006. 



Interim Results 

A comprehensive survey was administered to about 1 ,200 young people in the program and control 

groups an average of 21 months after they entered the study. Key findings from the survey include: 

• The program group was much more likely than the control group to have obtained a high 
school diploma or a General Educational Development certificate (GED) and to have 
earned college credits. For example, about 61 percent of the program group had earned a dip- 
loma or a GED, compared with 36 percent of the control group. 

• At the time of the survey, program group members were somewhat more likely to be 
engaged in productive activities. For example, 72 percent of the program group were working, 
in school or training, or in the military, compared with 66 percent of the control group. 

• Young people in the two groups were equally likely to have been arrested in the year prior 
to the survey, but the program group was less likely to have been convicted of a crime or 
to have engaged in certain delinquent acts. 

• There were few differences between groups in measures of physical or mental health. 

Differences between groups that were measured at an earlier point in the study had disappeared 
by the 21 -month point. 

These interim results are impressive, but longer-term follow-up will be critical to understanding the 

full story of the program’s effects. Results from a 36-month survey should be available by late 2010. 

iii 
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Preface 



High school dropout rates remain stubbornly high even as broad economic shifts con- 
tinue to erode opportunities for workers without postsecondary education. Finding ways to 
reengage high school dropouts and help them move forward in education and the labor market 
is a pressing national priority. The National Guard Youth ChalleNGe program has served more 
than 90,000 young high school dropouts since it was created in the early 1990s. The program, 
which now operates in more than half the states, uses an unusual model that combines an inten- 
sive residential phase with military-style discipline, a comprehensive focus on activities thought 
to promote positive youth development, and a postresidential mentoring program. 

The rigorous, random assignment evaluation of ChalleNGe described in this report is 
particularly timely, as the current administration has emphasized the importance of obtaining 
hard evidence on the effectiveness of federally funded programs. Early results from the evalua- 
tion, released in 2009, have already drawn attention to ChalleNGe, and Congress recently voted 
to increase the share of program costs that can be paid by the U.S. Department of Defense, a 
critical change for cash-strapped states. 

The evaluation represents a true collaboration. It is funded by a group of private founda- 
tions and the Department of Defense, and is led by MDRC, working closely with scholars from 
the MacArthur Foundation Research Network on Transitions to Adulthood. Twelve state Chal- 
leNGe programs agreed to participate, and 10 of them were eventually able to enroll about 
3,000 young people into the study sample. 

The interim results presented here are based on a survey administered to young people 
in the program and control groups an average of 21 months after they entered the study, when 
most were 18 or 19 years old. The program group members had completed the ChalleNGe pro- 
gram by that time. The survey found that young people who had access to ChalleNGe were 
much more likely than those in the control group to have obtained a high school diploma or a 
General Educational Development (GED) certificate. They were also somewhat more likely to 
be working, attending college, enlisted in the military, or engaging in other productive activities. 

While encouraging, these interim results are not the final word on ChalleNGe. Another 
survey, administered about 36 months after young people entered the study, is nearing comple- 
tion. It will be critical to see how the participants in the study have fared over a longer period, 
particularly because the survey was administered at a time when unemployment rates for disad- 
vantaged youth have reached crisis levels. Those results should be available by the end of 2010. 

Gordon Berlin 
President 
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Executive Summary 



Young people who drop out of high school face long odds of success in a labor market 
that increasingly values education and skills. Nationally, about 30 percent of high school 
freshmen do not graduate in four years; in the 50 largest U.S. cities, the dropout rate is closer to 
50 percent. 

This report presents interim results from a rigorous, ongoing evaluation of the National 
Guard Youth ChalleNGe Program, which aims to “reclaim the lives of at-risk youth” who have 
dropped out of high school and give them the skills and values to succeed as adults. ChalleNGe 
is an intensive residential program that currently operates in more than half the states. More than 
90,000 young people have completed the program since it was launched in the early 1990s. 
MDRC, a nonprofit, nonpartisan research organization, is conducting the evaluation in collabo- 
ration with the MacArthur Foundation Research Network on Transitions to Adulthood. Several 
private foundations and the U.S. Department of Defense are funding the evaluation. 



The ChalleNGe Program 

The ChalleNGe approach grew out of a project by the Center for Strategic and Interna- 
tional Studies (CSIS) in the late 1980s and early 1990s that sought to develop new approaches 
for out-of-school youth. Staff in the National Guard Bureau in the U.S. Department of Defense 
developed the specific program model. They had concluded that many existing programs for 
disadvantaged youth were “focused on the symptomatic behaviors without understanding and 
addressing the underlying causes” and “placed limited, if any, focus on the post-program 
phase.” Thus, they designed ChalleNGe to be: 

...an intervention, rather than a remedial program. We would deal with the 
symptoms and underlying causes in a construct that fully embraced a “whole 
person” change and readied the students for the post-program environment. 

We would arm them with the skills and experiences necessary to succeed and 
we would ensure there was “a way back” to mainstream society. 1 

In 1993, Congress funded a 10-site pilot of ChalleNGe. Funding was made permanent 
in 1998, and today there are 32 ChalleNGe programs in 27 states and Puerto Rico. 

States operate ChalleNGe programs under a Master Cooperative Agreement with the 
National Guard Bureau. Most states operate a single “100-bed” ChalleNGe program, serving a 



’Donohue (2008). 
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total of about 200 participants per year in two class cycles. A few states operate multiple 
programs or larger programs. The funding level for ChalleNGe — about $14,000 per participant 
— has not changed since the early 1990s. The federal government currently pays 60 percent of 
the cost of the state programs, and states pay the remaining 40 percent. 2 

Although there is considerable room to tailor the program model to local conditions, the 
basic structure of the ChalleNGe program is the same in all states. The program is open to 
young people between the ages of 16 and 18 who have dropped out of (or been expelled from) 
school, are unemployed, drug-free, and not heavily involved with the justice system. 3 The 
program is open to both males and females, though about 80 percent of the participants are 
male. There are no income-based eligibility criteria. 

The 17 -month program is divided into three phases: the Pre-ChalleNGe Phase (two 
weeks), the Residential Phase (20 weeks), and the Postresidential Phase (one year). During 
the first two phases (totaling 22 weeks), the participants live at the program site, often on a 
military base. 

The first phase, Pre-ChalleNGe, is a physically and psychologically demanding assess- 
ment and orientation period. Candidates are introduced to the program’s rules and expectations; 
learn military bearing, discipline, and teamwork; and begin physical fitness training. 

Candidates who complete Pre-ChalleNGe are formally enrolled in the program as “ca- 
dets” and move to the second phase. The curriculum for the 20-week Residential Phase is 
structured around eight core components that reflect current thinking about how to promote 
positive youth development: Leadership/Followership, Responsible Citizenship, Service to 
Community, Life-Coping Skills, Physical Fitness, Health and Hygiene, Job Skills, and Aca- 
demic Excellence. Cadets spend the largest share of each day in the education component. 
Most programs help participants prepare for the GED exam, but a few of them can offer a high 
school diploma. 

The program environment is described as “quasi-military”: The cadets are divided into 
platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject 
to military-style discipline. The daily schedule is highly structured with almost no “down 
time,” and the cadets are closely supervised by staff at all times. While ChalleNGe uses 



2 The ChalleNGe legislation was amended in late 2009, raising the maximum federal share of program 
costs to 75 percent. 

3 In order to be eligible for ChalleNGe, candidates must be 16 to 18 years of age and enter the program 
before their nineteenth birthday; a high school dropout/expellee; a citizen or legal resident of the United States 
and a resident of the state in which the program is conducted; unemployed; not currently on parole or on 
probation for anything other than juvenile status offenses, not serving time or awaiting sentencing, not under 
indictment or charged, and not convicted of a felony or a capital offense; and drug-free. 



ES-2 




military structure, discipline, facilities, and staff to accomplish its objectives, participation in 
the program is voluntary, and there are no requirements for military service during the program 
or afterward. 

Toward the end of the Residential Phase, the cadets work with staff to arrange a post- 
residential “placement.” Acceptable placements include employment, education, and military 
service. 



The cadets who successfully complete the Residential Phase move into the one -year 
Postresidential Phase, which involves a structured mentoring program. The ChalleNGe mentor- 
ing program is unusual, in that young people nominate their own mentors during the application 
process. ChalleNGe initiates the mentoring relationship partway through the Residential Phase, 
after the staff screen and train the mentors. The staff then maintain contact with both the 
program’s graduates and their mentors at least monthly during the Postresidential Phase to help 
solve problems and to monitor the youths’ progress. 



The ChalleNGe Evaluation 

The National Guard Bureau collects extensive data on program participation and partic- 
ipants’ outcomes. However, for some time, officials in the Department of Defense and many 
ChalleNGe program directors have been eager to obtain more rigorous data on what difference 
the program makes. The National Guard Bureau’s outcome data do not address this question 
because there is no way to know to what extent the outcomes that program participants or 
graduates achieve are actually attributable to their participation in ChalleNGe; the program 
serves relatively motivated young people who were determined to make a change and thus 
might make progress without ChalleNGe. Thus, in 2004, the officials and directors began 
working with MDRC and the Mac Arthur Foundation Research Network on Transitions to 
Adulthood to explore the possibility of conducting a random assignment evaluation of the 
program. Ultimately, the Department of Defense agreed to fund 20 percent of the evaluation, 
and MDRC raised the remaining 80 percent from private foundations. 

In 2005, 12 state ChalleNGe programs agreed to participate in the evaluation. These 
programs were not chosen randomly. Rather, there was an effort to identify programs that had 
stable staffing and that tended to receive more applicants than they could serve. 

The evaluation uses a random assignment research design in which a group of young 
people who applied to ChalleNGe and were invited to participate (the program group) is 
being compared over time with a second group (the control group) who applied to ChalleNGe 
and were deemed acceptable, but were not invited to participate. Random assignment was 
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conducted only during class cycles in which there were substantially more applicants than 
program slots. 

Because the study’s participants were assigned to the program group or the control 
group through a random process, one can be confident that any significant differences that 
emerge between the groups over time — for example, differences in educational attainment or 
employment rates — suggest that ChalleNGe was effective. These differences are described as 
impacts. 



Ultimately, random assignment was conducted for 18 class cycles across 10 programs 
in 2005 and 2006 (two of the sites that agreed to participate were unable to conduct random 
assignment because there were not substantially more qualified applicants than slots). About 
3,000 young people entered the study. 

A series of surveys is being administered to the program and control groups over time. 
The first survey, a very brief interview, was conducted about nine months after the young 
people entered the study — not long after participants in the program group had completed the 
residential phase of the program. The results from that survey, presented in an earlier report, 
were quite promising. 

This report presents the results from the second survey, a much more extensive inter- 
view conducted an average of 21 months after participants entered the study, after the postresi- 
dential phase of the program had ended. Just under 1,200 young people were interviewed, and 
the response rate was 79 percent. Most respondents were either 18 or 19 years old when they 
were interviewed. 



Results from the 21 -Month Survey 

• The program group was much more likely than the control group to 
have earned a GED and somewhat more likely to have obtained a high 
school diploma; the program group was also more likely to have earned 
college credits. 

As shown in the top panel of Table ES.l, about 61 percent of the program group re- 
ported on the survey that they had earned a high school diploma or a GED. The corresponding 
figure for the control group was 36 percent. The asterisks show that the difference between 
groups — about 24 percentage points — was statistically significant, meaning that it is very 
unlikely to have arisen by chance. As expected given the program model, many more program 
group members had earned a GED than a high school diploma, and the program’s impact was 
concentrated on GED receipt. Nevertheless, there was also a statistically significant increase in 
the percentage of sample members with a high school diploma. Program group members were 
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National Guard Youth ChalleNGe Program 

Table ES.l 

Impacts on Selected Outcomes from the 21-Month Survey 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 31 


Educational attainment (%) 

Earned high school diploma or GED certificate 


60.5 


36.4 


24 i *** 


0.000 


High school diploma 


22.0 


16.3 


5.7 ** 


0.013 


GED certificate 


48.3 


21.9 


26.5 *** 


0.000 


Earned any college credit 


24.8 


9.6 


15.1 *** 


0.000 


Current status 

Currently in (%) 

High school or GED prep 


16.1 


26.0 


_9 9 *** 


0.000 


College courses 


11.6 


7.0 


4 ^ *** 


0.009 


Job training 


7.2 


6.5 


0.7 


0.648 


Currently working (%) 


55.0 


50.1 


4.9 * 


0.093 


Average weekly earnings ($) 


209 


169 


39 *** 


0.009 


Currently enlisted in the military (%) 


10.9 


6.2 


4 7 *** 


0.004 


Currently involved in any of the above activities ’ 3 (%) 


72.1 


66.4 


5.7 ** 


0.033 


Has high school diploma or GED certificate and is currently 

involved in any of the above activities ' 3 (%) 45.5 


23.1 


22.4 *** 


0.000 


Crime and delinauencv 

In the past 12 months 
Arrested (%) 


26.3 


25.4 


0.9 


0.719 


Convicted (%) 


8.9 


13.2 


-4.2 ** 


0.020 


Any self-reported delinquency (%) 


62.6 


66.5 


-3.9 


0.168 


Number of delinquent incidents 


1.6 


1.9 


-0.3 ** 


0.014 


Health (%) 

Overall health good or excellent 


71.0 


69.4 


1.6 


0.548 


Obese c 


9.8 


11.3 


-1.5 


0.425 


Serious psychological distress 11 


5.4 


5.1 


0.3 


0.844 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

“Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear on the 
table): 2.827, 2.773, 2.264, 2.358,and 2.734. 

b This measure includes any employment, school or GED programs, vocational training, military 
activities, or any residential programs (not listed separately above). 

C A person is defined as obese if his or her Body Mass Index (BMI) is 30 or higher. 

d The K 6 scale is the sum of the responses to six questions asking how often a respondent experienced 
symptoms of psychological distress. The scale ranges from 0 to 24. A score of 13 points or more on the K 6 
scale is considered an indication of serious psychological distress. 
http://www.hcp.med.harvard.edu/ncs/k 6 _scales.php. 
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more than twice as likely to report that they had earned at least one college credit, though 
relatively few people in either group reported that they were currently in college when inter- 
viewed (second panel). 

• Young people in the program group were more likely to be engaged in 
productive activities. 

The second panel of Table ES.l describes sample members’ activities at the point that 
they responded to the survey. The table shows that control group members were more likely to 
be in high school or a GED preparation program, and that program group members were more 
likely to be in college, working, or in the military. 

Overall, about 72 percent of the program group was engaged in one of these productive 
activities, compared with 66 percent of the control group. A separate measure examined the 
percentage of each group that had completed a diploma or GED and was engaged in a produc- 
tive activity, a rough indicator of progress in the transition to adulthood. The program group 
was about twice as likely to be in this status. 

• There was no significant difference between groups in the percentage 
arrested in the year prior to the survey; the program group, however, 
was less likely to have been convicted of a crime and to have engaged in 
certain types of delinquency. 

As shown in Table ES.l, relatively large percentages of both groups reported involve- 
ment with the criminal justice system or delinquent activities. For example, about one-fourth of 
each group reported being arrested in the previous 12 months, and almost two-thirds of each 
group reported that they had engaged in at least one of 13 specific delinquent activities (for the 
most part, the delinquent activities were relatively minor, such as fighting). Although there were 
no significant differences between groups in either of these measures, the program group was 
less likely to report a conviction and also reported fewer delinquent activities. 

• There were no systematic differences between the groups on measures of 
current physical or mental health; most respondents in both groups re- 
ported being in relatively good health. 

At the nine-month point, the program group reported better health and less obesity. 
However, as shown in Table ES.l, those differences were no longer evident at the 21 -month 
point. There was also no difference between groups in a general measure of psychological 
distress. Other analyses, not shown in the table, found few differences in living arrangements or 
other measures of well-being. 
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Conclusions and Next Steps 

The results presented here constitute the second chapter in the unfolding story of the 
ChalleNGe program’s impacts. Twenty-one months after entering the study, at age 18 or 19, 
young people in the program group, who had access to ChalleNGe, were substantially more 
likely to have earned a high school diploma or a GED and were also more likely to be engaged 
in productive activities such as working or attending school. Those in the program group were 
also less likely to have been convicted of a crime and appeared to have engaged in few delin- 
quent activities. 

While quite promising, the 21 -month results also suggest some reasons for caution. As 
might be expected, many of the differences measured between the groups at 21 months were 
smaller than the corresponding differences at the nine-month point. Moreover, because many 
members of the control group were attending high school or GED preparation classes when 
they were interviewed, the program group’s advantage in high school completion may continue 
to narrow over time. Relatively few program group members were in college, so it is not yet 
clear how many ChalleNGe graduates will use their high school credentials to obtain further 
education or training that may qualify them for higher-paying jobs. 

Results from the 36-month survey, scheduled for late 2010, will fill in the next chapter 
in this important story. The next report will also expand on analyses that are presented in 
preliminary form in this report — for example, analysis of the program’ s impacts on measures 
of psychosocial development, and an examination of the program’ s impacts for different types 
of youth or different ChalleNGe programs. 
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Introduction 



Young people who drop out of high school face long odds of success in a labor market 
that increasingly values education and skills. Nationally, about 30 percent of high school 
freshmen do not graduate in four years; in the 50 largest U.S. cities, the dropout rate is closer to 
50 percent. 1 Although many of those who drop out eventually graduate or, more often, earn a 
General Educational Development certificate (GED), a long delay may place them at a serious 
disadvantage in competing for jobs and going on to college. 2 Moreover, a significant number of 
young people become profoundly “disconnected” from both school and work. In 2007, before 
the current recession, only 60 percent of 16-to-24-year-old high school dropouts worked at all 
during the year. Almost one in 10 young male dropouts were incarcerated. 3 

This report presents interim results from an ongoing evaluation of the National Guard 
Youth ChalleNGe Program, which is designed to “reclaim the lives of at-risk youth” who have 
dropped out of high school and give them the skills and values to succeed as adults. 4 ChalleNGe 
is an intensive residential program that currently operates in more than half the states. About 
90,000 young people have completed the program since it was launched in the early 1990s. 
MDRC, a nonprofit, nonpartisan research organization, is conducting the evaluation in collabo- 
ration with the MacArthur Foundation Research Network on Transitions to Adulthood. Several 
private foundations and the U.S. Department of Defense are funding the evaluation. 5 



Earlier Evaluations of Youth Programs and the 
Origins of ChalleNGe 

During the last three decades, a number of rigorous evaluations have assessed programs 
targeting disadvantaged youth. Some studies tested programs that primarily served youth who 
were at risk but were still in the regular K-12 education system, while others studied “second- 
chance” programs for out-of-school youth, typically high school dropouts. 



'Swanson (2008). However, Roy and Mishel (2008) argue that graduation rates may be higher than re- 
ported in many recent studies. 

“One national study tracked students who were in the eighth grade in 1988. About 20 percent of the stu- 
dents dropped out of high school at least once. Among the dropouts, 63 percent earned a high school diploma 
(19 percent) or a GED (44 percent) by 2000, eight years after their scheduled graduation date (Hurst, Kelly, 
and Princiotta, 2004). 

3 Sum, Khatiwanda, and McLaughlin (2009). 

4 Adapted from the National Guard Youth ChalleNGe Program’s mission statement. 

5 The study is funded by Bill & Melinda Gates Foundation, Charles Stewart Mott Foundation, The Edna 
McConnell Clark Foundation, The John D. and Catherine T. MacArthur Foundation, The MCJ Foundation, 
The Robert Wood Johnson Foundation, and The William and Flora Hewlett Foundation. 
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The overall record from the studies of second-chance programs is mixed. On the one 
hand, several programs significantly increased the percentage of young people who earned a 
GED or another credential. In addition, some of the programs — particularly those that offered 
participants subsidized jobs — generated significant increases in employment or earnings in the 
short term. Others led to decreases in arrests or other measures of criminal justice involvement. 
On the other hand, however, none of the studies that followed participants for more than a 
couple of years found lasting improvements in economic outcomes. Some of the studies did not 
report or collect long-term data while, in other cases, early gains in earnings faded over time. 6 

In response to disappointing evaluation results in the 1980s, experts argued that youth 
programs should not just address problems or “deficits,” but rather should promote “positive 
youth development.” Specifically, they recommended that programs should go beyond educa- 
tion and training to expose young people to activities, settings, and relationships that are thought 
to promote healthy development. ChalleNGe, along with programs such as YouthBuild, is part 
of this movement. 7 

The ChalleNGe model grew out of a project by the Center for Strategic and Internation- 
al Studies (CSIS) in the late 1980s and early 1990s that sought to develop new approaches for 
out-of-school youths. The project’s final report concluded that aspects of the military structure 
could be beneficial for disadvantaged youth. 8 Many others have made this argument, most 
recently in a report by the Brookings Institution, which concluded that “the United States 
military enjoys a well-deserved reputation for its ability to reach, teach, and develop young 
people who are rudderless, and for setting the pace among American institutions in advancing 
minorities.” 9 

The CSIS report also concluded that the National Guard, with its strong community 
service mission, was well suited to operate a program for at-risk young people. Staff in the 
National Guard Bureau in the U.S. Department of Defense developed the program model for 
ChalleNGe. They had concluded that many existing programs for disadvantaged youth were 
“focused on the symptomatic behaviors without understanding and addressing the underlying 
causes” and “placed limited, if any, focus on the post-program phase.” Thus, they designed 
ChalleNGe to be: 

...an intervention, rather than a remedial program. We would deal with the 
symptoms and underlying causes in a construct that fully embraced a “whole 



6 Bloom (2010). 

7 YouthBuild programs serve youth ages 16 to 24. Participants work toward their GED or high school dip- 
loma while learning skills by building affordable housing. For more information, see www.youthbuild.org. 

8 Cullinan, Eaves, McCurdy, and McCain (1992). 

9 Price (2007). 
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person” change and readied the students for the post-program environment. 

We would arm them with the skills and experiences necessary to succeed and 
we would ensure there was “a way back” to mainstream society. 10 

In 1993, Congress funded a 10-site pilot of ChalleNGe. Funding was made permanent 
in 1998, and today there are 32 ChalleNGe programs in 27 states and Puerto Rico. 



The ChalleNGe Model 

States operate ChalleNGe programs under a Master Cooperative Agreement with the 
National Guard Bureau. Most states operate a single “100-bed” ChalleNGe program, serving a 
total of about 200 participants per year in two class cycles (starting in January and July). A few 
states operate multiple programs or larger programs. For example, during the study period, 
Georgia operated two 200-bed programs, Illinois operated a single 400-bed program, and 
Mississippi operated a single 200-bed program. 

The funding level for ChalleNGe — about $14,000 per participant — has not changed 
since the early 1990s (in real terms, per-participant funding has fallen by about a th ir d during 
that time). Thus, the typical 100-bed program has an annual budget of about $3 million. Since 
1998, the federal government has paid 60 percent of the cost of the state programs, and states 
have paid the remaining 40 percent. 1 1 

Programs typically assemble the state share of the funding from several different state 
and local sources. In some cases, a local school district supplies teachers. In some programs, 
small nonprofit organizations raise modest sums to support extra program activities, such as a 
graduation stipend, team uniforms, a graduation dinner dance, and a yearbook. In interviews, 
many program directors reported that state budget cuts in recent years had forced them to reduce 
enrollment targets, require staff to work uncompensated overtime, cut back on staff training, and 
eliminate program extras that could not be recovered from nonprofit or parent organizations. 12 

Although there is considerable room to tailor the program model to local conditions, the 
basic structure of the ChalleNGe program is the same in all states. The program is open to 
young people between the ages of 16 and 18 who have dropped out of (or been expelled from) 



“’Donohue (2008). 

n The ChalleNGe legislation was amended in late 2009, raising the maximum federal share of program 
costs to 75 percent. 

l2 The field research was mostly conducted in 2006, well before the current recession. States may be hav- 
ing more difficulty securing matching funds in the current economic environment. 
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school, are unemployed, drug-free, and not heavily involved with the justice system. 13 The 
program is open to both males and females, though about 80 percent of the participants are 
male. There are no income-based eligibility criteria. 

As shown in Figure 1, the 17-month program is divided into three phases: the Pre- 
ChalleNGe Phase (two weeks), the Residential Phase (20 weeks), and the Postresidential Phase 
(one year). During the first two phases (totaling 22 weeks), the participants live at the program 
site, often on a military base. 

The first phase, Pre-ChalleNGe, is a physically and psychologically demanding assess- 
ment and orientation period. Candidates are introduced to the program’s rules and expectations; 
learn military bearing, discipline, and teamwork; and begin physical fitness training. 

Candidates who complete Pre-ChalleNGe are formally enrolled in the program as 
“cadets” and move to the second phase. The curriculum for the 20-week Residential Phase is 
structured around eight core components that reflect current thinking about how to promote 
positive youth development: Leadership/Followership, Responsible Citizenship, Service to 
Community, Life-Coping Skills, Physical Fitness, Health and Hygiene, Job Skills, and 
Academic Excellence. Cadets spend the largest share of each day in the education component. 
Most programs help participants prepare for the GED exam, but a few of them can offer a 
high school diploma. 

The structure of the residential part of the program is designed to minimize some of the 
potentially negative effects of placing many at-risk young people together in a program setting 
— sometimes referred to as “deviant peer influences” or “peer contagion.” 14 The program 
environment is described as “quasi-military”: The cadets are divided into platoons and squads, 
live in barracks, have their hair cut short, wear uniforms, and are subject to military-style 
discipline. The daily schedule is highly structured with almost no “down time,” and the cadets 
are closely supervised by staff at all times. While ChalleNGe uses military structure, discipline, 
facilities, and staff to accomplish its objectives, participation in the program is voluntary, and 
there are no requirements for military service during the program or afterward. 

Toward the end of the Residential Phase, the cadets work with staff to arrange a post- 
residential “placement.” Acceptable placements include employment, education, and military 
service. 

13 In order to be eligible for ChalleNGe, candidates must be 16 to 18 years of age and enter the program 
before their nineteenth birthday; a high school dropout/expellee; a citizen or legal resident of the United States 
and a resident of the state in which the program is conducted; unemployed; not currently on parole or on 
probation for anything other than juvenile status offenses, not serving time or awaiting sentencing, not under 
indictment or charged, and not convicted of a felony or a capital offense; and drug-free. 
l4 Dodge, Dishion, and Lansford (2007). 
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National Guard Youth ChalleNGe Program 
Figure 1 

ChalleNGe Program Phases 




17.5 Months 



The cadets who successfully complete the Residential Phase move into the one -year 
Postresidential Phase, which involves a structured mentoring program. The ChalleNGe mentor- 
ing program is unusual, in that young people nominate their own mentors during the application 
process. ChalleNGe initiates the mentoring relationship partway through the Residential Phase, 
after the staff screen and train the mentors. The staff then maintain contact with both the 
program’ s graduates and their mentors at least monthly during the Postresidential Phase to help 
solve problems and to report on the youths’ progress. 

The structured Postresidential Phase distinguishes ChalleNGe from most residential 
programs for youth. The purpose of this phase is to help ChalleNGe participants with the 
difficult task of maintaining the new attitudes and behaviors they have learned in the Residential 
Phase when they return to their communities, families, and friends. Mentoring is a promising, 
low-cost strategy for providing guidance and support for young people. 



The ChalleNGe Evaluation 

The National Guard Bureau collects fairly extensive data on program participation and 
participants’ outcomes. These data are used for program management and to inform an annual 
report to Congress. However, for some time, officials in the Department of Defense and many 
ChalleNGe program directors have been eager to obtain more rigorous data on what difference 
the program makes. The National Guard Bureau’s outcome data do not address this question 
because there is no way to know to what extent the outcomes that program participants or 
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graduates achieve are attributable to their participation in ChalleNGe; the program serves 
relatively motivated young people who might make progress on their own, without ChalleNGe. 
Thus, in 2004, the officials and directors began working with MDRC and the MacArthur 
Foundation Research Network on Transitions to Adulthood to explore the possibility of con- 
ducting a random assignment evaluation of the program. Ultimately, the Department of Defense 
agreed to fund 20 percent of the evaluation, and MDRC raised the remaining 80 percent from 
private foundations. 

In 2005, 12 state ChalleNGe programs (almost half the state programs in existence at 
the time) agreed to participate in the evaluation. These programs were not chosen randomly. 
Rather, there was an effort to identify programs that had stable staffing and that tended to 
receive more applicants than they could serve, a prerequisite for conducting a random assign- 
ment evaluation. Table 1 shows some basic information about each of the 12 programs that 
agreed to participate. In most states, the annual graduation goal for 2005 (the year the study 
began) was split across two class cycles. 

Research Design 

The ChalleNGe evaluation uses a random assignment research design in which a group 
of young people who applied to ChalleNGe and were invited to participate (the program group) 
is being compared over time with a second group (the control group) who applied to ChalleNGe 
and were deemed acceptable, but were not invited to participate. 

Because the study’s participants were assigned to one group or the other through a ran- 
dom process, one can be confident that any significant differences that emerge between the 
groups over time — for example, differences in educational attainment or employment rates — 
can be attributed to ChalleNGe. These differences are described as impacts. 

Although random assignment is generally considered the most reliable way to assess 
the effectiveness of social programs, the design can cause ethical concerns if the creation of a 
control group reduces the number of people who receive program services. Thus, for the 
ChalleNGe evaluation, MDRC worked with the Department of Defense and the participating 
programs to develop a random assignment process that aimed to ensure that the evaluation 
would not reduce the number of young people who received ChalleNGe’ s services. MDRC’s 
Institutional Review Board reviewed and approved the design. 

Under this design, the participating ChalleNGe programs recruited and screened appli- 
cants more or less as usual and identified a pool of applicants who met all eligibility criteria 



6 




National Guard Youth ChalleNGe Program 
Table 1 



Information on Participating Programs 



State 


Location 


First Year of Operation 


Annual Graduation Target (2005) 


Arizona 


Queen Creek 


1993 


224 


California 


Camp San Luis Obispo 


1998 


212 


Florida 


Camp B landing 


2001 


220 


Georgia 


Fort Gordon 


2000 


200 


Illinois 


Rantoul 


1993 


675 


Michigan 


Battle Creek 


1999 


200 


Mississippi 


Camp Shelby 


1994 


400 


New Mexico 


Roswell 


2001 


200 


North Carolina 


Salemburg 


1994 


200 


Texas 


Galveston 


1999 


200 


Virginia 


Camp Pendleton 


1994 


200 


Wisconsin 


Fort McCoy 


1998 


200 



SOURCE: National Guard Bureau (2005). 



and were considered acceptable. 15 Random assignment was conducted for a particular class 
cycle only if the number of acceptable applicants in the pool was at least 25 greater than the 
number of available program slots. 16 In other words, the programs would have had to turn 
away some applicants for these class cycles even without the evaluation. To facilitate the 
evaluation, states agreed to use a random process to decide which qualified applicants to accept 
(at least one state already used a random process but most did not). Also, to preserve the 
integrity of the design, applicants who were assigned to the control group were not allowed to 
reapply for later class cycles. 

The original plan was to conduct random assignment for two class cycles per program 
and to obtain a sample size of about 2,500 youths (1,250 per group). As shown in Table 2, 
however, there were many class cycles in which the number of applicants was too small to 
allow random assignment to take place. For example, in the first class cycle of 2006, only six 



15 The Department of Defense authorized a modest amount of funding to support enhanced recruitment 
efforts by the programs that participated in the evaluation. 

16 Although the programs often refer to the number of available “beds,” in fact, the number of available 
slots is often determined not by physical space but by funding for staff. Typically, the programs are funded and 
staffed to graduate a certain number of participants per cycle (100 in most programs). During the study period, 
program managers told MDRC how many applicants they needed to accept in order to meet the graduation 
target, assuming normal patterns of attrition. Random assignment was conducted if the number of qualified 
applicants was at least 25 greater than the number needed to meet the graduation goal. 
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National Guard Youth ChalleNGe Program 
Table 2 



Use of Random Assignment Across ChalleNGe Sites 
by Program, Year, and Class Cycle 



Program 


2005 


2006 


2007 


Cycle 2 


Cycle 1 


Cycle 2 


Cycle 1 


AZ 




O 


O 




CA 




O 


X 


O 


FL 




X 


X 




GA 




X 


X 




IL 




X 


X 




MI 


X 


X 


X 




MS 




O 


X 


X 


NC 




X 


X 




NM 




o 


X 


O 


TX 




X 


X 




VA 




o 


o 




WI 




o 


X 


O 



SOURCE: MDRC random assignment database. 

NOTES: "X" indicates that random assignment was conducted. "O" indicates that random 
assignment was attempted but not conducted because the number of applicants was too 
small. 



of the 12 participating programs were able to conduct random assignment. 17 This occurred 
primarily because many programs tended to recruit only enough applicants to fill the available 
program slots. 

Ultimately, random assignment was conducted for 18 class cycles across 10 programs. 
One program (Michigan) conducted random assignment three times, six programs conducted 
random assignment twice, and three programs conducted random assignment once. Two of the 
sites that agreed to participate (Arizona and Virginia) were unable to conduct random assign- 
ment because they did not have more qualified applicants than slots during the study period. 
The total sample size (3,074) exceeded the original goal but was more heavily weighted toward 



l7 Only the Michigan program attempted to conduct random assignment in 2005. For the first cycle of 
2007, only states that had previously conducted random assignment one time attempted to conduct it again. 
The six programs that had already conducted random assignment more than once were excluded, as were the 
two programs that had been unable to conduct random assignment. 




the program group than originally intended (the sample includes 754 in the control group and 
2,320 in the program group). 18 

Data Sources 

The evaluation draws data from several sources: 

• Baseline questionnaire. Just before they entered the study, the applicants 
completed a two-page questionnaire that was inserted into the ChalleNGe 
application packet in the study sites. 19 These data provide a snapshot of the 
study participants just before they were randomly assigned to the ChalleNGe 
program or to the control group. 

• Program participation data. MDRC obtained information from the Chal- 
leNGe Data Management and Reporting System (DMARS), the national 
Web-based program tracking system used by all ChalleNGe programs. 

• Site visits. Members of the evaluation team conducted two-day visits to 
each of the 10 programs that conducted random assignment. Each visit in- 
cluded structured interviews with both program staff and participants. The 
study’s earlier report provided detailed information on the program’s im- 
plementation. 

• Follow-up surveys. MDRC conducted a competition and selected Westat, 

Inc., to administer follow-up surveys for the study. Westat administered the 
first survey, a short questionnaire, by phone or in person an average of nine 
months after members of the program and control groups entered the study. 
Results from that survey were discussed in an earlier report. A more exten- 
sive survey was administered an average of 21 months after the young 
people entered the study; the results from that survey are presented in this re- 
port. A third survey wave, approximately 36 months after they entered the 
study, was completed in early 2010. 

The 21 -month survey, the focus of this report, was administered from July 2007 to Au- 
gust 2008. It targeted 1,507 sample members (916 in the program group and 592 in the control 
group). The sample was selected at differing rates across sites and random assignment status to 

18 Sample sizes in the individual sites are relatively small, so most of the analysis will pool results from all 
the sites. Technically, the pooled results do not represent the overall impact of ChalleNGe nationally because 
the study sites were not chosen randomly. 

l9 The applicants also signed a consent form to participate in the study at this point. If they were under age 
18 , a parent or guardian also signed the form. 
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minimize the variance of estimated impacts when sites are weighted equally in the analysis. A 
total of 1,196 sample members (736 program group and 460 control group) completed the 
survey, for an overall response rate of 79 percent. On average, respondents were 18.5 years old 
when they responded to the survey. 

A response bias analysis compared the baseline characteristics of those who completed 
the 21 -month survey with the full research sample. A separate analysis compared the fielded 
survey sample with those who completed the survey. A few significant differences emerged in 
each of these cases. However, some differences are expected by chance, and tests predicting 
response or treatment status were consistent with there being chance differences as indicated by 
the test statistics. In addition, the respondent sample for this survey was compared with that of 
the earlier nine -month survey, in order to understand any comparisons of impacts between the 
two surveys. The two survey samples overlap sufficiently to make such comparisons. See 
Appendix A for more information on these analyses. 

The analysis of the survey data presented in this report controls for a number of base- 
line characteristics of the survey sample, including age, gender, race, whether the sample 
member was interested in ChalleNGe because he or she wanted to join the military, whether he 
or she lived in a two-parent household, and highest grade completed. Weighting was used in the 
full sample and subgroups to adjust for site size (so each of the 10 sites contributes equally to 
the results), survey response rates, and program versus control ratios. Alternate weights were 
created for the full sample that did not incorporate response rates, but these did not produce any 
differences in key outcomes. 



Characteristics of the Study’s Participants 

Table 3 presents selected information from the two-page survey that all sample mem- 
bers completed when they applied for ChalleNGe. These data provide a “snapshot” of the 
applicants as they entered the study. In general, the data indicate that ChalleNGe is serving a 
diverse group of high school dropouts. 20 



20 Although ChalleNGe serves young people from 16 to 18 years old, the youngest applicants — those 
under age 16 and a half — were excluded from the evaluation; in other words, they were not subject to random 
assignment. Specifically, ChalleNGe applicants were excluded from random assignment if they would have 
been under 17 years old on the last day of the Residential Phase of the class cycle for which they applied. 
Owing to this rule, the characteristics of the participants in the study do not necessarily match those of all the 
young people who participated in the programs during the cycles when random assignment occurred. 
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National Guard Youth ChalleNGe Program 
Table 3 



Selected Characteristics of ChalleNGe Sample Members at the 
Time of Random Assignment, Full Sample 





All 


Characteristic (%) 


Sites 


Gender 




Male 


84.1 


Female 


15.9 


Age in years 




16 


36.8 


17 


52.2 


18 


11.0 


Race/ethnicity a 




Hispanic 


14.4 


White 


41.3 


Black 


40.1 


Other 


4.2 


Lives with 




Both biological parents 


23.3 


Mother only 


37.1 


Father only 


6.4 


One parent and a stepparent 


20.9 


No parental figures 


10.9 


Other combination 


1.6 


Anyone in household receives public assistance 


29.5 


Highest grade completed 




8th grade or lower 


14.2 


9th grade 


31.4 


10th grade 


38.2 


1 1th grade 


15.6 


12 th grade 


0.6 


Usual grades received in school 




Mostly As and Bs 


4.0 


Mostly Bs and Cs 


17.2 


Mostly Cs and Ds 


39.5 


Mostly Ds and Fs 


48.7 


Has/had Individual Education Plan (IEP) 


30.4 


Ever suspended from school 


82.3 


Ever arrested 


31.1 


Ever convicted 


16.3 


Sample size 


3,074 


(continued) 
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Table 3 (continued) 



Characteristic (%) 


All 

Sites 


Who first suggested you should apply for ChalleNGe? 


Yourself 


28.0 


A relative 


47.9 


A school official 


13.7 


The justice system 


6.3 


Reasons for applying to ChalleNGe? 


Want a high school diploma/GED certificate 


81.3 


Want to go to college/get more training 


44.5 


Want to get a job 


38.8 


Want to join the military 


30.7 


Want to get life on track 


76.9 


Overall health very good or excellent 


66.0 


Taking any medication 


20.8 


Overweight (BMI 25-29) b 


20.8 


Obese (BMI 30+) b 


11.8 


Ever drink alcohol or use drugs 


36.2 


Sample size 


3,074 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: Calculations for this table used all available data for ChalleNGe sample members who 
completed a BIF. 

Data for the "All Sites" column represent an average of all 3,074 sample members. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

Distributions may not add to 100 percent where categories are not mutually exclusive. 
a Race/ethnicity categories were constructed by counting as Hispanic those who checked both 
Hispanic and black or white. None of these sample members are counted as multiracial and 
grouped in the "other" category. 

b Body Mass Index (BMI) is a measure of body fat based on height and weight that applies to 
both adult men and women. BMI is calculated by dividing a person's weight by his or her height 
squared. A person is defined as overweight if his or her BMI is between 25 and 29.9. A person is 
defined as obese if his or her BMI is 30 or higher. 
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As shown in Table 3, most of the participants in the study were 17 years old at the 
point when they entered the study, and about 84 percent are male. 21 Roughly equal proportions 
described themselves as white (41 percent) or Aftican-American/black (40 percent); most of 
the rest described themselves as Hispanic. Almost all are U.S. citizens and were bom in the 
United States, and only about 3 percent reported having any children of their own (not shown 
in the table). 

Only 23 percent of the sample members lived with both biological parents when they 
entered the study; another 21 percent lived with a parent and a stepparent. More than 40 percent 
lived in a single-parent household (most commonly with their mother), and about 1 1 percent 
lived with no parent or stepparent. Less than one-third of sample members reported that their 
household received any public assistance, indicating that the ChalleNGe population is not, in 
general, extremely low income (though it is possible that some sample members were not aware 
that their household received public assistance). 

As expected, the participants in the study had not done well in school before leaving. 
About half reported that their grades had been mostly Ds and Fs, and more than 80 percent 
reported that they had been suspended from school at least once. Nearly one -third reported that 
they currently or previously had an Individual Education Plan, which indicates special educa- 
tion status. 

About two-thirds of the study’s participants characterized their health as very good or 
excellent. On the other hand, about one-third were either obese or overweight. About one-third 
reported that they had used drugs or alcohol, though sample members may have underreported 
their drug use if they believed that the baseline survey was actually part of the program’s 
application process. 



Summary of Earlier Findings 

The evaluation’s first report described the implementation of ChalleNGe in the study 
sites, discussed why young people had enrolled in the program, and presented results from a 
survey administered about nine months after sample members entered the study. 22 



“'in some cycles, it was not possible to include female applicants in the random assignment pool, because 
the programs needed to accept all or nearly all female applicants, as female staff had already been hired to 
work with them. Thus, the percentage of females in the research sample is slightly lower than the percentage of 
females in the programs. Typically, about 20 percent of graduates nationwide are female. 

22 Bloom, Gardenhire-Crooks, and Mandsager (2009). 
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Program Implementation and Participation in ChalleNGe 

Field research visits to all 10 of the participating programs highlighted significant varia- 
tion across sites in the program environment, approaches to recruitment and discipline, and 
other elements of the program. In interviews, staff also highlighted a number of implementation 
challenges. For example, many programs reported that they were grappling with funding 
shortages and were dealing with the absence of experienced staff who were National Guard 
members deployed to Iraq or Afghanistan. Nevertheless, the basic structure of the program was 
quite similar from site to site, all of the programs were implementing all of the core elements of 
the program, and staff were generally highly committed and professional. Box 1 includes 
selected quotes from ChalleNGe participants who were interviewed by the research team (these 
quotes are drawn from the study’s first report). 

As shown in Table 4, data from the program’s national management information sys- 
tem (MIS) show that about 83 percent of the young people who were assigned to the study’s 
program group actually started the program (that is, they showed up and registered); others 
may have changed their mind about participating after they were invited or showed up to the 
program and failed a drug screen. Nearly 70 percent of the program group completed the pre- 
ChalleNGe phase and formally enrolled, and a little over half graduated from the residential 
phase. The graduation rate among enrollees was about 78 percent, close to the national average 
for this time period. According to the MIS, more than 80 percent of program graduates had 
contact with their mentor after completing the residential phase. On the 21 -month survey, 
about 7 1 percent of respondents who were ever assigned a mentor reported that they were still 
in touch with their mentor, even though the formal program had ended by the time the inter- 
views took place. 23 

In order to preserve the integrity of the random assignment research design, the impact 
analysis includes all program and control group members, including the program group mem- 
bers who did not participate in ChalleNGe or dropped out before completing the program. It 
also includes the small number of control group members who participated in ChalleNGe. 24 

Results from the Nine-Month Survey 

The nine-month survey was quite brief, but the results were fairly striking. The program 
group was more than four times more likely than the control group to have a high school 



~ 3 Owing to the program’s unusual mentoring model, in which the young people nominated their own 
mentors during the application process, most of the sample members already knew their mentor before they 
enrolled in ChalleNGe. This may help to explain why such a large proportion of sample members were still in 
touch with their mentors well after the formal program had ended. 

24 A total of eight control group respondents (1.7 percent) enrolled in the ChalleNGe program. 
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Boxl 



Selected Quotations from ChalleNGe Participants 
Reasons for coming to ChalleNGe: 

. . .1 hated high school. Just everything about it. 1 just hated it. I hated the classes; I hated the 
people... The main reason I came here was to stop, like, all the partying scene and every- 
thing; I was getting really sick of that. . .1 realized that there was nothing going for me there, 
so I needed a change. 

About Pre-ChalleNGe: 

It was hell. Yeah, it was rough, man. Right when they got in front of us, “Get off the bus! 
Get off the bus!” Yelling at us, getting our hair cut, screaming all the time. I was like 
“. . .Crap, I’ve got to get out of here!” But the sergeants helped me get through it. . .telling 
me, “You've got to stay! You’ve got to get through it. We’ll get through it together.” 

After Pre-ChalleNGe 1 felt 1 accomplished something. . .1 called my mom and I bursted out 
crying. 1 said, “Mom, I made it!” and Mom said, “I knew you had it in you. Don’t give 
up!” “I won’t, Mom.” That made me feel great. 

On discipline in ChalleNGe: 

1 used to get smoked a lot..before, when I wouldn’t listen. 1 was hardheaded. I didn’t listen 
to nobody. [The staff would say] “Get down [to do pushups].” [And 1 would say], 
“No”. . .but they’re doing it so you could learn. . .1 learned the hard way. 

Contrasting ChalleNGe teachers to regular high school teachers: 

“I’m getting my paycheck. If I teach you, I teach you. If you don’t want to listen, then hey, 
I’m not going to by to help you.” That’s how it was. Now, it’s like, “I can help you. I been 
reaching out to you. Like, just grab the string. I can help you; I’m going to pull you up. Just 
don’t let go, because I’m going to help you make it to the top.” 



diploma or a GED. The program group was also significantly more likely to be working and to 
be in college, and less likely to have been arrested since entering the study. There were also 
statistically significant improvements in obesity, overall health, and self-efficacy. 

Despite these very positive results, the report interpreted the findings cautiously. It 
pointed out that the survey had been conducted very early in the follow-up period — in fact, 
most program group respondents were still participating in the Postresidential Phase. Moreover, 
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National Guard Youth ChalleNGe Program 
Table 4 



Selected Graduation Data 
Program Group Members, All Sites 



Outcome (%) 


Full 

Program Group 


Among 
Those Registered 


Among 
Those Enrolled 


Registered 


82.5 


100.0 




Enrolled 


67.9 


82.3 




Graduated 


52.7 


63.9 


77.7 


Sample size (total = 2,320) 


2,320 


1,913 


1.575 



SOURCE: MDRC calculations using data from the ChalleNGe Data Management and Reporting 
System (DMARS). 



the report noted that several earlier studies had cast doubt on the value of the GED in the labor 
market, finding that GED holders earn significantly less than high school graduates. Studies 
have shown that postsecondary education pays off as much for GED holders as for regular high 
school graduates, but that only a small minority of GED holders complete even one year of 
postsecondary education. 25 These data may help to explain why youth programs that substan- 
tially increased GED receipt did not lead to longer-term gains in employment or earnings. 

For all of these reasons, the report noted that the results of later follow-up surveys — 
both the 21-month survey reported here and the 36-month survey that will be completed in 2010 
— would be critical to understanding the full story of the program’s impacts. 



25 Tyler (2005). 
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Results from the 21 -Month Survey 



This section uses data from the 21 -month survey to examine the impacts of ChalleNGe 
on sample members’ educational attainment and their current activities, including work, 
education, training, and military enlistment. As discussed earlier, two of the program’s key 
goals are to help young people further their education and get a foothold in the labor market. 
Beyond education and job training, the program was designed to foster a “whole person 
change” and help youth find “a way back” to society. Thus, the survey also examined a wide 
range of measures of health, crime and delinquency, psychosocial development and civic 
engagement. 



Education and Employment 

Overall, the survey data suggest that, as a result of their participation in ChalleNGe, the 
program group had better educational and employment outcomes than the control group. 
Specifically, a greater proportion of the program group had earned a high school diploma or a 
General Educational Development certificate (GED), and the program group was more likely to 
be working full time, taking college courses, or enlisted in the military. Among the specific 
findings: 

• The program group was much more likely than the control group to 
have obtained a GED and somewhat more likely to have earned a high 
school diploma. 

As Table 5 shows, 61 percent of the program group reported having a high school dip- 
loma or a GED, compared with 36 percent of the control group. The difference of 24 percentage 
points is statistically significant, meaning that such a result would be very unlikely to have 
arisen from an ineffective program. The difference between groups was even larger — 36 
percentage points — at the nine-month follow-up point. The impact on GED receipt remained 
fairly stable, but the impact on high school diploma attainment grew smaller between the two 
survey waves. Based on the significantly higher percentage of the control group reporting 
current enrollment in high school or GED preparation courses at the 21 -month point (see Table 
6), the gap between groups may shrink further over time. 

Interestingly, as shown in Table 5, about 10 percent of the program group reported hav- 
ing both a high school diploma and a GED certificate. This is an unusual combination, since a 
GED is usually seen as an alternative to a high school diploma. Further analysis showed that the 
program group members who reported having both a diploma and a GED are concentrated in a 
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National Guard Youth ChalleNGe Program 
Table 5 



Impacts on Education and Training 



Outcome (%) 


Program 

Group 


Control 

Group 


Impact 


P- Value 3 


Earned high school diploma or GED certificate 


60.5 


36.4 


24 i *** 


0.000 


High school diploma 


22.0 


16.3 


5.7 ** 


0.013 


GED certificate 


48.3 


21.9 


26.5 *** 


0.000 


Both high school and GED certificate 


9.7 


1.7 


7 g *** 


0.000 


Earned any college credit 


24.8 


9.6 


15.1 *** 


0.000 


Received college degree 


0.3 


0.0 


0.3 


0.280 


Ever received vocational training 


29.7 


22.9 


6.8 ** 


0.010 


Received trade license/training certificate 


19.1 


15.8 


3.3 


0.150 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 

NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * 
= 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

“Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear on the 
table): 2.827, 2.773, 1.441, and 2.264. 



few ChalleNGe sites. One of those sites is an alternative high school, and in another, an ar- 
rangement with the state legislature allows young people who complete ChalleNGe and pass the 
GED exam to receive a state high school diploma. 

Finally, Table 5 shows that the program group was more than twice as likely as the con- 
trol group to have earned at least one college credit. The vast majority (80 percent) of sample 
members who had received college credit reported that it was outside of ChalleNGe or another 
residential program (not shown in the table). Therefore, the difference between the groups does 
not reflect only the college courses available at some of the ChalleNGe sites. There was no 
significant difference between groups in the percentage reporting that they had earned a voca- 
tional certificate. 
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National Guard Youth ChalleNGe Program 
Table 6 

Impacts on Current Activities and Periods of Inactivity 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 3 


Current status 

Currently in (%) 


High school 


6.5 


13.1 


-6.6 *** 


0.000 


GED prep 


10.2 


14.1 


_3 9 ** 


0.042 


College courses 


11.6 


7.0 


4 6 *** 


0.009 


Job training 


7.2 


6.5 


0.7 


0.648 


Currently involved in any of the above activities (%) 


31.6 


34.7 


-3.1 


0.262 


Currently working (%) 


55.0 


50.1 


4.9 * 


0.093 


Current hourly wage under $6 


2.9 


4.3 


-1.4 


0.208 


Current hourly wage between $6 and $7.99 


12.9 


16.1 


-3.2 


0.123 


Current hourly wage between $8 and $9.99 


22.6 


18.2 


4.5 * 


0.064 


Current hourly wage $10 or more 


9.1 


6.2 


2.8 * 


0.078 


Currently working full time (%) 


43.7 


38.8 


4.9 * 


0.090 


Current average weekly earnings c ($) 


209 


169 


39 *** 


0.009 


Currently enlisted in the military (%) 


10.9 


6.2 


4 7 *** 


0.004 


Active Army 


4.2 


1.7 


2.5 ** 


0.018 


Army National Guard 


4.1 


1.7 


2.4 ** 


0.021 


Other 


2.2 


1.4 


0.7 


0.374 


Currently involved in any of the above activities 11 (%) 


72.1 


66.4 


5.7 ** 


0.033 


Has high school diploma or GED certificate and is currently 


involved in any of the above activities' 3 (%) 


45.5 


23.1 


22.4 *** 


0.000 


Periods of inactivity (%) 


Since random assignment, idle for one or more months 


46.1 


52.4 


-6.2 ** 


0.036 


Idle for 1-2 months 


20.4 


16.0 


4.5 * 


0.054 


Idle for 3-1 1 months 


22.8 


29.5 


-6 7 *** 


0.010 


Idle for a year or more 


2.9 


6.9 


_4 Q *** 


0.001 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21 -month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear on the 
table): 1.688,2.734. 

b This measure includes any employment, school or GED programs, vocational training, military 
activities, or any residential programs (not listed separately above). 

c Weekly earnings averages include zeroes for respondents who were not employed. 
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• The program group was significantly more likely than the control 
group to be working full time, earning higher wages, and enlisted in 
the military. 

Table 6 describes sample members' current activities at the time of the interview. As 
noted earlier, the survey was completed in mid 2008, before the sharp rise in the unemployment 
rate nationally. It shows that about half of the entire sample reported that they were currently 
employed, either part or full time. However, there is a statistically significant difference be- 
tween the program group and the control group in both overall employment and full-time 
employment. 1 Again, these differences are somewhat smaller than at the nine-month point. 

Among those currently working for pay, the program group was earning significantly 
higher wages. As the salary categories in the middle of Table 6 illustrate, a higher percentage of 
the control group was earning close to the minimum wage, from $6 to $8 an hour, in their 
current job. 2 Meanwhile, the program group was more likely to be earning more than $8 per 
hour. Further reinforcing these differences in employment and wages, average weekly earnings 
at the time of the survey were $209 for the program group and $169 for the control group. 
Weekly earnings take into account the employment rates, total work hours, and hourly wages 
for both groups. 3 With a larger percentage of program group members working full time and a 
larger percentage earning somewhat higher hourly wages, the average difference of nearly $40 
per week is consistent with the other employment outcomes. 

In view of the National Guard sponsorship and quasi-military environment of Chal- 
leNGe, it is not surprising that those who participated in the program were nearly twice as 
likely to be enlisted in the military at the time of the survey (10.9 percent versus 6.2 percent). 
The majority of enlisted sample members were either in the active Army or Army National 
Guard. However, it is worth noting that more than 30 percent of the research sample reported 
at baseline that they were interested in ChalleNGe in part because they wanted to join the 
military (see Table 3). In other words, an interest in possible military enlistment preceded any 



'Full-time employment is defined here as 30 hours or more per week. 

2 The federal minimum wage during the time of the interviews was primarily $5. 85/hour; it rose to 
$6.55/hour during the last month of interviews. For those receiving wages under S6/hour, the jobs appeared to 
be employment that includes tips, such as waiting tables, and has significantly lower minimum wages. If 
respondents were working in two jobs, the hourly wage data refer to the job in which they worked the most 
hours per week. 

3 This average weekly earnings calculation includes all repotted current jobs by the sample member and 
includes earnings of zero dollars for those not currently working. Less than one percent of the entire sample 
reported weekly earnings of more than $1,000 and the majority of those working reported weekly earnings 
between $120 and $600. An analysis of weekly earnings was conducted, removing the questionably high 
repotted earnings, and results were similar to those reported in Table 6. 
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active engagement with ChalleNGe for about a third of the sample. In addition, it is possible 
that military participation is underreported in the survey. The survey firm reported difficulty 
in completing interviews with sample members in the military, particularly those who were 
deployed. To address this, future reports on ChalleNGe will likely include analysis of addi- 
tional data sources to further understand the military enlistment patterns and performance of 
this sample. 

• The control group was more likely to be in high school or a GED prep- 
aration course at the time of the survey, while the program group was 
more likely to be enrolled in college courses. 

Based on the higher rates of high school diploma or GED receipt in the program group, 
it is not surprising that the control group was more likely to be enrolled in high school or a GED 
preparation program. Similarly, it makes sense that a greater proportion of program group 
members have gone on to college course work. About 12 percent of the program group was 
currently enrolled in college courses, a statistically significant 4.6 percentage point increase 
over the control group. As noted earlier, however, there was a larger difference between the two 
groups in the percentage who had earned any college credits (see Table 5). It is also notable that 
the number currently in college is much smaller than the percentage who ever earned a credit. 
This suggests that many sample members may have started college and then left, a common 
pattern, particularly among community college students. 4 

• The control group reported longer periods of inactivity since random 
assignment. 

At the time of survey, program group members were more likely to be engaged in a 
productive activity. For example, 72 percent of the program group was in school or training, 
working, or in the military, compared with 66 percent of the control group. A broader measure, 
designed to measure progress in the transition to adulthood, shows that 45 percent of the 
program group and 23 percent of the control group were engaged in a productive activity and 
had a high school diploma or a GED (this measure obviously reflects the large difference in 
high school completion discussed earlier). 

In an attempt to further understand how sample members had been spending their time, 
the survey asked about time “basically not doing anything” since random assignment. The 
survey defined this inactivity (or “idleness”) as time when the respondent was not working, in 
school, in a training program, or in the military. As shown in Table 6, approximately half the 
sample reported one or more months of inactivity since entering the study. However, the control 



4 Horn and Weko (2009). 
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group reported significantly more months of “doing nothing.” This is illustrated by the higher 
percentages of the control group reporting inactivity for three or more months since random 
assignment. The program group was significantly more likely to report being idle for only one 
or two months (20 percent of the program group versus 16 percent of the control group). These 
reports of inactivity reinforce the higher rates of participation in “productive” activities and 
educational attainment by the program group. 5 



Crime and Delinquency 

As noted earlier, high school dropouts (particularly males) are at high risk of becoming 
involved with the criminal justice system. Moreover, about one- t hird of the research sample 
reported at least one arrest or conviction prior to random assignment (see Table 3). By engaging 
young people and putting them on a more positive path, ChalleNGe hopes to reduce criminal 
activity in the future. The survey data suggest a small to moderate reduction in crime and 
delinquency attributable to the program, consistent across self-reports of both involvement in 
the criminal justice system and broader categories of delinquency. Specifically: 

• The program and control groups were equally likely to report being ar- 
rested and charged with a crime in the past year, but the control group 
was more likely to have been convicted of a crime. 

As shown in the top panel of Table 7, about a quarter of the survey sample — similar 
across groups — reported having been arrested and charged with a crime in the previous year. 
However, significantly more control group members reported being convicted of a crime. It is 
not clear what caused the lower conviction rates, but one hypothesis is that the program group 
had better contacts to help them navigate the judicial system and avoid conviction. For exam- 
ple, about 18 percent of the program group reported being charged with a public order offense 
(see Appendix Table B.l), with approximately a third (6 percent) ending in a conviction; for 
the control group, 13 percent reported a public order charge, with more than half (7 percent) 
ending in a conviction. In breaking down the convictions by type of offense, there was a 
significant difference between the two groups only in the category of property crime convic- 
tions. 6 Property offenses include burglary, larceny, motor vehicle theft, arson, fraud, forgery, 
embezzlement, and others. 



5 The survey also asked whether sample members in both groups had been in “residential youth programs” 
(other than ChalleNGe). About 10 percent of each group reported that they had participated in such a program, 
with the Job Corps mentioned most frequently. 

6 For categorization of offenses, see Langan and Levin (2002). 
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National Guard Youth ChalleNGe Program 
Table 7 



Impacts on Delinquency and Criminal Activity in the Last 12 Months 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 


Self-reported arrests/convictions (%) 


Arrested 


26.3 


25.4 


0.9 


0.719 


Charged with any crime 


25.0 


24.7 


0.3 


0.917 


Convicted of any crime 3 


8.9 


13.2 


-4.2 ** 


0.020 


Convicted of a violent crime 


1.4 


1.2 


0.2 


0.748 


Convicted of a property crime 


1.5 


4.1 


-2.6 *** 


0.005 


Convicted of a drug crime 


1.4 


1.9 


-0.5 


0.536 


Convicted of a public order crime 


5.6 


6.7 


-1.1 


0.428 


Self-reported delinauencv b 


Any incidents (%) 


62.6 


66.5 


-3.9 


0.168 


Any violent incidents (%) 


54.0 


57.3 


-3.3 


0.263 


Any property incidents (%) 


27.2 


35.1 


_7 9 *** 


0.004 


Number of incidents 


3.3 


4.1 


_Q g *** 


0.003 


Number of violent incidents 


2.0 


2.3 


-0.4 ** 


0.035 


Number of property incidents 


0.9 


1.3 


_0 4 *** 


0.005 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21 -month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

““Conviction categories from Langan and Levin (2002). 
b See full list of types of delinquency in Box 2. 



Comparing these findings with the earlier survey wave is difficult. At the time of the 
nine-month survey, arrest and conviction rates were lower overall, and the control group 
members were significantly more likely to have been arrested or convicted since random 
assignment. However, this represented a shorter time span (an average of nine months versus a 
full year). In addition, a large percentage of the program group had spent much of that nine- 
month period in the ChalleNGe residential program, where they were closely monitored and 
tightly scheduled. 
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• A majority of respondents in both groups reported being involved in 
some form of delinquency in the previous year; the program group re- 
ported committing fewer total acts of delinquency. 

The second panel of Table 7 refers to 13 survey questions that asked if the sample 
member had committed a series of specific delinquent acts in the past 12 months. 7 These 
activities ranged widely in severity and type of offense; see Box 2 for a full list of the items and 
response categories and Appendix B for detailed information on which offenses sample mem- 
bers reported. While there is likely overlap between these questions and offenses that may have 



Box 2 




Types of Self-Reported Delinquent and Criminal Behavior 


Survey respondents were asked how often they had committed a variety of offenses in the 
last 12 months, with the option of refusing to answer or answering “never,” “once,” “twice,” 
“three or four times,” “five or more times,” or “don’t know.” The types of self-reported 


offenses arc listed below. 




Property incidents 


Violent incidents 


• Deliberately damaged property 


• Got into a serious physical fight 


• Stole something worth more than $50 


• Got into a fight with another group 


• Stole something worth less than $50 


• Hurt someone badly enough to need 


• Entered a house he or she was not supposed 


medical attention 


to enter in order to steal something 


• Used or threatened to use a weapon 


• Drove a car without owner’ s permission 

• Bought, sold, or held stolen property 


to get something 


• Used someone else’ s credit card/bank 


Other incidents 


card without permission 
• Deliberately wrote a bad check 


• Sold marijuana or other drugs 


SOURCE: Haynie (2001). 





resulted in arrests and convictions discussed above, these questions give a broader view of the 
potential delinquency of the sample. Overall, Appendix B shows that fewer than 10 percent of 



7 Haynie (2001). 
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respondents reported committing most of the offenses listed in Box 2. However, about half 
reported that they had been in a serious physical fight, and about one in four reported that they 
had hurt someone badly enough that the person needed bandages or medical treatment. 

The control group was somewhat more likely to report engaging in at least one of the 
delinquent activities (67 percent versus 63 percent), though the difference between groups is not 
quite statistically significant. However, the control group reported a significantly greater 
number of incidents than the program group, an average of 4.1 incidents versus 3.3 incidents, 
respectively. 

In an effort to further understand the types of offenses being reported and compare them 
with the conviction categories in the top panel, the self-reported offenses were divided into two 
categories. The first category includes eight questions that represent property offenses, ranging 
from “deliberately damaged property” to “stole something.” The second category, violent 
offenses, includes four questions about fighting, mainly, and the use of a weapon (see Box 2). In 
line with the differences in convictions in the top panel, a significantly higher percentage of 
control group members (35 percent versus 27 percent) reported at least one property-related 
incident. The control group also reported a greater number of violent incidents. As noted earlier, 
incidents categorized as violent were overwhelmingly related to fighting, rather than use of a 
weapon, so it is not clear whether they would have led to the arrests or convictions in the top 
panel of Table 7. 



Health and Other Outcomes 

• No dear health differences emerged between the two groups. 

Table 8 shows that there were no statistically significant differences between groups in 
measures of health and sexual activity. Most respondents in both groups reported that they were 
in good or excellent health, and only about 10 percent of each group were obese (defined as 
having a Body Mass Index (BMI) of 30 or higher). 8 About 5 percent of each group were 
experiencing serious psychological distress according to a widely used six-question scale. 

At the time of the nine-month survey, statistically significant differences were seen both 
for the self-rating of overall health and for obesity. The healthier scores for the program group 
at that time were likely attributable to the physical activity and the diet in the recently completed 
ChalleNGe residential program. 



8 The BMI is a measure of body fat based on height and weight that applies to both adult men and women. 
It is calculated by dividing a person’s weight by his or her height squared. A person is considered overweight if 
his or her BMI is between 25 and 29.9. Obesity is defined as a BMI of 30 or higher. 
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National Guard Youth ChalleNGe Program 
Table 8 



Impacts on Health, Sexual Activity, and Use of Substances 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 


Physical and mental health 


Overweight 2 (%) 


25.8 


24.8 


1.0 


0.698 


Obese 2 (%) 


9.8 


11.3 


-1.5 


0.425 


Overall health either excellent or good (%) 


71.0 


69.4 


1.6 


0.548 


Psychological distress score b 


4.9 


5.1 


-0.2 


0.358 


Serious psychological distress 13 (%) 


5.4 


5.1 


0.3 


0.844 


Sexual activity (%) 


Sexually active 


88.8 


86.1 


2.7 


0.159 


Always uses birth control 


53.3 


52.9 


0.3 


0.918 


Uses birth control sometimes or most of the time 


28.2 


25.7 


2.5 


0.347 


Never uses birth control 


10.3 


11.1 


-0.7 


0.682 


Drug and alcohol use (%) 


Binge drinking (5 or more drinks in a row) in the last 14 days 
Frequent marijuana use (10 or more occasions) 


2.8 


4.7 


-2.0 * 


0.076 


in the last 12 months 


22.5 


25.2 


-2.7 


0.282 


Ever used other illegal drugs 0 

Frequent illegal drug use (6 or more occasions) 


24.1 


23.1 


1.0 


0.684 


in the last 12 months 


5.6 


4.4 


1.3 


0.327 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a Body Mass Index (BMI) is a measure of body fat based on height and weight that applies to both adult 
men and women. BMI is calculated by dividing a person's weight by his or her height squared. A person 
is defined as overweight if his or her BMI is between 25 and 29.9. A person is defined as obese if his or 
her BMI is 30 or higher. 

b The K6 scale is the sum of the responses to six questions asking how often a respondent experienced 
symptoms of psychological distress. The scale ranges from 0 to 24. A score of 13 points or more on the K6 
scale is considered an indication of serious psychological distress. 
http://www.hcp.med.harvard.edu/ncs/k6_scales.php. 

c Other illegal drugs" refers to drugs other than marijuana. Examples given in the survey question 
included: LSD or other psychedelic drugs, cocaine, crystal meth, amphetamines, barbiturates, heroin, and 
performance-enhancing substances like anabolic steroids. 
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There is mixed evidence about the program’s impacts on substance 
abuse. 



In the last panel of Table 8, there is evidence of a small beneficial program effect on 
binge drinking. However, for both groups the reported rates are lower than might be expected 
for this age group. The binge drinking rates for the general population are reported in the 
Monitoring the Future study as 8 percent for eighth-graders, 16 percent for tenth-graders, and 25 
percent for twelfth-graders. 9 Unlike binge drinking, rates of marijuana and other illegal drug use 
are higher in the ChalleNGe sample than in the general population. For example, more than 20 
percent of the survey sample reported frequent marijuana use, while less than 4 percent of 
twelfth-graders reported this type of marijuana usage in Monitoring the Future. 10 However, the 
rates of marijuana and other illegal drug use are comparable for the program and control groups. 

• Program group members were more likely to be living on their own and 
also more likely to be a parent (or pregnant). 

The survey also measured rates of marriage, parenthood, and living independently, all 
potentially related to the transition to adulthood. As shown in Table 9, somewhat more than half 
of each group reported living with their parents at the time of the survey; at the nine-month 
point, about three-quarters of each group reported living with their parents. 

Program group members were somewhat more likely to be living in their own home or 
apartment at the 21 -month point (19 percent versus 15 percent). However, the two groups were 
about equally likely to be married or living with a partner. Program members were somewhat 
more likely to report being pregnant or having one or more biological children. This result is 
somewhat puzzling since there were no impacts on measures of sexual activity or use of birth 
control. Moreover, there is no evidence that program group members were more likely to be 
living on their own with a child. 



Life-Coping, Leadership, and Group Interaction Skills 

Cultivating interpersonal and coping skills necessary to deal with the challenges of the 
transition to adulthood, as well as leadership and group interaction abilities, are shared objec- 
tives of ChalleNGe programs — a part of daily life and a focus of many specific activities. All 



’Johnston, O'Malley, Bachman, and Schulenberg (2008). 

"’Johnston, O’Malley, Bachman, and Schulenberg (2008). Students in the Monitoring the Future survey 
were asked how frequently they had used marijuana over various periods of time. A total of 3.6 percent of 
those surveyed repotted using marijuana 10 to 19 times over the previous year. 
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National Guard Youth ChalleNGe Program 
Table 9 



Impacts on Living, Marital, and Parental Status 



Outcome (%) 


Program 

Group 


Control 

Group 


Impact 


P- Value 


Livins and marital status 


Living at parents' home 


57.8 


60.5 


-2.7 


0.346 


Living in own home or apartment 


19.0 


15.0 


4.0 * 


0.073 


Living in someone else's home 


17.4 


19.8 


-2.5 


0.283 


Other living arrangement 2 


5.8 


4.5 


1.3 


0.315 


Never married and not living with partner 


85.4 


83.8 


1.6 


0.459 


Married or living with a partner 


13.6 


15.4 


-1.8 


0.391 


Divorced or separated 


1.0 


0.8 


0.2 


0.718 


Parental status 


Has at least one child or is pregnant 


17.2 


13.6 


3.6 * 


0.091 


Has a child 


15.9 


12.6 


3.3 


0.113 


Pregnant 


1.5 


1.5 


-0.1 


0.93 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * 
= 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

“"Other living arrangement" includes: living in a group home or homeless shelter, not living 
anywhere (homeless), or any other arrangement not specifically listed. 



of these qualities are thought to be important for growth and healthy adaptation during young 
adulthood and the years beyond. 

The 21 -month survey included a series of questions on psychosocial development that 
were either selected or adapted from existing measures or were unique items developed and 
piloted by the MacArthur Research Network on Transitions to Adulthood. This section focuses 
on a subset of those questions addressing life -coping, leadership, and group skills. 

Table 10 reports on a preliminary analysis with single questions selected because they 
were: (1) directly salient to the core components of ChalleNGe and (2) had existing compara- 
tive data from other research/evaluations (see Appendix C for comparative data means). 



28 







National Guard Youth ChalleNGe Program 
Table 10 



Impacts on Leadership and Life- Coping Skills 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 3 


Life-copins skills ' 3 

Has learned about setting priorities 


3.5 


3.5 


0.0 


0.419 


Has learned about organizing time and not 
putting things off 


3.6 


3.5 


0.1 * 


0.060 


Has learned how to better control temper 


3.5 


3.4 


0.1 ** 


0.013 


Has become better at dealing with fears 
and worries 


3.6 


3.6 


0.0 


0.522 


Has tried new ways of acting around people 


3.3 


3.2 


0.1 


0.162 


Leadership/followership and group skills ' 3 

Has observed how others have solved problems 
and learned from them 


3.6 


3.6 


0.0 


0.751 


Has learned the challenges of being a leader 


3.7 


3.5 


0.2 *** 


0.000 


Has had the experience of others counting 
on him/her 


3.6 


3.6 


0.0 


0.316 


Tries to present his/her ideas without 
criticizing others 0 


3.5 


3.3 


0.1 * 


0.063 


Encourages different points of view without 
worrying about agreement 0 


3.7 


3.5 


0.1 ** 


0.040 


Tries to consider all points of view or possible 
options before forming an opinion or making 
a decision 0 


4.0 


4.0 


0.0 


0.595 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21 -month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear on the 
table); .0399. 

b Except where noted, all responses are on a 1-4 scale, with 1 indicating a negative response and 4 
indicating a positive response. 

c Responses on these items are on a 1-5 scale, with 1 indicating a negative response and 5 indicating a 
positive response. 
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Comparative data sources include items from the Youth Experience Survey, version 2.0" and a 
national evaluation of the Americorps program. The majority of these items are on a 1 to 4 
scale, with the final three on a 1 to 5 scale. For all, a 1 is a negative response and a 4 (or 5) 
indicates a positive response. 

Future reports will move beyond the single-item analysis to evaluate these psychosocial 
measures using composite measures assessing a broader array of developmental outcomes. See 
Appendix C for a discussion of some of these preliminary exploratory analyses. 

Life-Coping Skills 

Through a combination of classroom activities, a structured living environment, and 
mentoring relationships, ChalleNGe aims to help young people develop individual strategies 
and coping mechanisms for managing their personal lives and dealing with emotions. Partici- 
pants are provided daily models of managing their time, as well as effective coping and emo- 
tional and behavioral regulation skills. The young people must learn to cope with negative 
emotions not by suppressing or dwelling on them, but by reappraising them. For example, in 
situations that might induce anger and frustration, ChalleNGe encourages participants to think 
about ways that will help them stay calm, find meaning in what they are going through, and 
allow them to move forward in pursuing their goals. 12 These are all-important skills related to 
moving forward with life plans and coping strategies. Research indicates that such aspects of 
planfulness discriminate between vulnerable adolescents who go on to achieve (both academi- 
cally and in their careers) and those who do not. 13 

Table 10 shows that there were some statistically significant differences between the 
program and control groups on responses to questions that focus on organizing time and not 
putting things off. In addition, there was a statistically significant difference between the 
program and control group means (3.5 and 3.4, respectively) in emotion regulation, specifically 
learning how to control one’s temper. These tentative findings seem particularly important, 
considering some of the emotional and coping vulnerabilities of ChalleNGe participants before 
they entered the program. Further investigation should also determine whether increased time- 
and anger-management skills may translate to more adaptive long-term outcomes in work, 
school, and interpersonal realms. 

No statistically significant differences were found in related life-coping items that 
evaluate skills in setting priorities, dealing with fears and worries, and trying out new ways of 
acting with people. Even though there was no evidence of statistically significant differences, 

"Hansen and Larson (2005). 

"John and Gross (2004, 2007). 

13 Masten et al. (2004). 
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the trends were in predictable directions, with participants in the program group more likely to 
endorse higher levels of competency on all of these life-coping skill items. 

To contextualize these findings, an analysis compared responses of ChalleNGe sample 
members with those of eleventh-graders used for instrument development of the Youth Expe- 
rience Survey 2.0 (YES), the survey the items were adapted from. 14 ChalleNGe participants 
were more likely to give positive responses (stronger positive endorsements of items) than 
participants in the comparative YES sample (see Appendix C). The specificity of ChalleNGe as 
a single pointed intervention (versus a range of YES community activities with potentially 
varying and more diluted impacts across the sample) might partially account for this finding. 

Leadership and Group Skills 

Daily life in a structured group environment aims to foster cadets’ abilities to serve as 
good leaders and harmonious and respectful group members. This is the focus of many of their 
roles and daily responsibilities, both in the barracks and the classroom. Participants were 
evaluated on multiple items related to this core tenet of the intervention. 

As shown in Table 10, a higher percentage of program group members reported that 
they learned the challenges of being a leader. These results were statistically significant. 
However, no significant differences were found on items asking whether they had had the 
experience of others counting on them or whether they had learned from how others around 
them solve problems. 

Another potential impact of ChalleNGe relates to how well young people do in cultivat- 
ing positive group interactions. The survey included items from the Americorps evaluation 
about having the interpersonal skills to foster a group environment where everyone’s point of 
view is heard, valued, and considered in making decisions. Statistically significant differences 
were found between program and control participants’ means on questions that evaluate the 
ability of young people to present ideas without criticizing the ideas of others as well as encour- 
aging different points of view without worrying about agreement. However, no differences were 
found between groups on then evaluating the ideas and opinions of all in order to make a 
decision. Means of individual and grouped items are comparable, although a bit lower (less 
strongly endorsed), than means on the same items from youth in the Americorps evaluation (see 
Appendix C). 



14 Hansen and Larson (2005). 
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Civic Engagement 

The core components of the ChalleNGe program are organized with the explicit goal of 
affecting such outcomes as responsible citizenship and service to the community. Measures 
discussed below were incorporated into the survey to evaluate aspects of these core civic-related 
program components. As with the measures of psychosocial development discussed in the 
previous section, the civic engagement questions will be the subject of additional analysis in 
later reports. 

As shown in Table 1 1, even though civic development is a core component of the Chal- 
leNGe program, the evaluation yielded fewer statistically significant differences between 
program and control groups than anticipated. 15 

The most promising finding was that reports of past civic engagement behavior were 
significantly higher in the program group than in the control group. These items ask respondents 
whether they had ever reported a crime that they witnessed, attended a meeting where there was 
a discussion of an issue in their communities, attended religious services regularly, voted in 
elections, kept informed about current issues, worked with a group to solve a problem in their 
community, or contacted a newspaper, radio, or TV talk show to express his or her opinion on 
an issue. With the exception of attending religious services, most of the items tap behavior that 
teenagers would be less likely to engage in simply by virtue of their age. This might be an 
important positive effect of the program if former cadets continued to engage in such communi- 
ty and civic activities, because it also would mean that they would be maintaining contact with 
social and community organizations. 

There was a marginally positive effect of the program on cadets’ justice-oriented citizen 
behaviors (that is, the importance they attached to challenging inequalities or protesting to bring 
about needed change). It is tempting to interpret this as a positive effect, but since there were no 
other similar statistically significant positive program effects, it is important to note that this 
result may simply be due to chance. If cadets’ endorsement of conventional citizenship behavior 
(that is, the importance they attached to assisting people who need help, following rules and 
laws, being concerned about national, state, and local issues, and working with local organiza- 
tions and government to solve problems) had also increased, there might be reason to conclude 
that the program had affected their civic goals. But since these results do not support such 
conclusions, caution should be exercised in evaluating this finding. 



l5 Survey questions in this section were provided by the Mac Arthur Network. Items shown in the table are 
a combination of existing scales in this field and additional ones created by factor analysis work. See Appendix 
D for a description of, and more information about, these exploratory analyses and factor loadings. 
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National Guard Youth ChalleNGe Program 
Table 11 



Impacts on Civic Engagement 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 


Civic engagement 










Collective civic efficacy 2 


1.89 


1.98 


-0.09 ** 


0.029 


Social trust 2 


3.64 


3.66 


-0.01 


0.767 


Social responsibility to be a good person 2 


4.49 


4.45 


0.05 


0.247 


Belief in civil justice 2 


4.19 


4.14 


0.05 


0.281 


Sense of civic competence 13 


3.12 


3.07 


0.04 


0.179 


Past civic engagement 2 


0.39 


0.35 


0.04 *** 


0.004 


Belief in American promise 2 


2.73 


2.74 


-0.01 


0.920 


Justice-oriented citizen 13 


3.53 


3.46 


0.06 * 


0.092 


Conventional citizen scale* 3 


3.44 


3.38 


0.05 


0.117 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21 -month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

Rounding may cause slight discrepancies in sums and differences. 

“Average of item responses on a 1 (low) to 5 (high) scale. 
b Average of item responses on a 1 (low) to 4 (high) scale. 
c Average of responses to a series of seven Yes (1) or No (0) items. 



Interestingly, participants in the control group reported a higher level of collective civic 
efficacy than those in the program group. This aspect of civic engagement was measured based 
on the respondent’s strength of endorsing beliefs that, by working with fellow members of his 
or her community, he or she could be effective in addressing community problems. The 
following items formed this scale: 

• Thinking about problems you see in your community, how much difference 
do you believe that people working together as a group can make in solving 
those problems? 

• Thinking about problems you see in your community, how much difference 
do you believe you personally can make in working to solve those problems? 
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• I feel that individuals can be more effective in solving community problems 
if they work with a group rather than work alone. 

• If you want to solve a problem in your community, it’s better to work on it 
alone than to get bogged down in a group. 

The decline in collective civic efficacy associated with the program may reflect the 
realization that cadets gain from their involvement in a community service project — that it is a 
challenge for a group to work together to solve a community problem. A decline of this kind is 
not unheard of in the literature on service learning, insofar as youth feel more optimistic about 
capacities to solve community issues before they actually have any experience trying to do so. 16 

It is less clear why ChalleNGe had no impact on the cadets’ sense of civic competence, 
that is, their belief that, if they knew about a problem in their community, they would feel 
confident that they could create a plan to address it, get others to care about it, and express their 
views in public. It is possible that the measures of civic competence do not match the activities 
in the program; that is, the kinds of service activities cadets engaged in may not provide them 
with opportunities to develop these types of leadership skills. A second possibility is that only a 
small subset of the cadets (for example, those with the disposition to lead a civic project) gained 
confidence in their capacities to do so. 



Results for Subgroups 

Some programs are more or less effective for specific subsets of the population they 
serve. Data on such subgroup effects can inform decisions about how to target scarce program 
resources. An exploratory analysis examined whether the impacts of ChalleNGe differ for three 
sets of subgroups defined by characteristics measured at the point young people entered the 
study: age, high school academic performance, and previous involvement with the justice 
system. Tables 12, 13, and 14 show selected key outcomes for these subgroups; additional 
outcomes can be found in Appendix E. Future reports will include additional subgroup analysis. 

During field visits, ChalleNGe staff reported that younger participants can be more dif- 
ficult to work with because they are less focused and mature. Moreover, participants who are 16 
or 17 years old when they finish the residential phase tend to have fewer options for postpro- 
gram placements. Table 12 shows selected impacts for sample members who were 16 years old 



l6 Kahne and Westheimer (2006); Levine (2008); Styvertsen, Flanagan, and Stout (2007). 
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at the time of enrollment and for those who were 17 or 18. 17 The last column (labeled “H- 
Stars”) shows whether the differences in impacts between the two subgroups are statistically 
significant. Although some of the outcomes for the control group are quite different for the two 
age groups (for example, 40 percent of the older youth had completed a diploma or GED, 
compared with 27 percent of the younger youth), it does not appear that the program impacts 
differed systematically by age. 

For the second subgroup analysis, shown in Table 13, sample members were split be- 
tween those who reported a pre -random assignment arrest or conviction and those who did not. 
One hypothesis was that involvement in ChalleNGe might help to “straighten out” young 
people who had been involved with the justice system. Again, outcomes look quite different 
across the groups. As expected, sample members with a pre-random assignment arrest or 
conviction were more likely to have a post-random assignment arrest or conviction. However, 
there are no clear patterns of differential impacts across the groups. For example, it appears that 
ChalleNGe had a larger impact on military enlistment for the high-delinquency subgroup 
(perhaps because completing the program helped program group youth who had prior records 
gain access to the military), but program group members in that subgroup were also more likely 
than their control group counterparts to report engaging in delinquent acts. 

A third subgroup analysis examined whether ChalleNGe’ s impacts differed for young 
people who were experiencing the greatest academic difficulty before they enrolled. There are 
several baseline measures that might be used to create academic performance subgroups, but the 
analysis in Table 14 divides the research sample according to sample members' self-reported 
grades in high school (“mostly Ds and Fs” versus better grades) There were competing hypo- 
theses regarding the program’ s potential effectiveness for these two groups. On the one hand, 
ChalleNGe may have larger impacts for young people who would fare worse on their own. On 
the other hand, the program’s focus on the GED might create problems for young people who 
are unable to reach that milestone during the residential phase. No clear patterns of different 
program impacts emerged between the groups. Future reports may include further analysis 
using different definitions of this subgroup. 



I7 As noted earlier, youth who were under 16.5 years at the time of enrollment were not included in the 
study. 
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Table 14 

Selected Impacts, by Academic Performance 
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0.000 (presented in the order in which they appear on the table): "Mostly Ds and Fs": 4.250, and 4.042. "Mostly Better than Ds and Fs": 3.899, and 
3.834. 

b This measure includes any employment, school or GED programs, vocational training, military activities, or any residential programs (not listed 
separately above). 




Conclusions and Next Steps 

The results presented in this report constitute the second chapter in the unfolding story 
of the ChalleNGe program’ s impacts. Twenty-one months after they entered the study, at age 18 
or 19, young people in the program group, who had access to ChalleNGe, were substantially 
more likely to have earned a high school diploma or a GED and were also more likely to be 
engaged in productive activities such as working or attending school. The program group was 
also less likely to have been convicted of a crime and appeared to have engaged in fewer 
delinquent activities. Together, these results suggest that the program group is better positioned 
to move forward in education, employment, and other critical realms. 

While quite promising, the 21 -month results also suggest some reasons for caution. As 
might be expected, many of the differences measured between the groups at 21 months were 
smaller than the corresponding differences at the nine-month point. In some areas, such as 
health, the earlier differences disappeared altogether. Moreover, because many control group 
members were attending high school or GED preparation classes when they were interviewed, 
the program group’s advantage in high school completion may continue to narrow over time. 
Relatively few program group members were in college, so it is not yet clear how many 
ChalleNGe graduates will use their high school credentials to obtain further education or 
training that may qualify them for higher-paying jobs. Finally, as noted earlier, some other 
evaluations of youth programs have found impacts on GED receipt that did not translate into 
longer-term improvements in economic outcomes. 

Results from the 36-month survey, scheduled for late 2010, will fill in the next chapter 
in this important story. The analysis of that survey will also include a more in-depth examina- 
tion of impacts for subgroups and sites, and impacts on measures of psychosocial development 
and civic engagement. Those results will provide important clues about the program’ s longer- 
term impacts but, even then, the results may not be definitive. Most of the sample members 
will be only 20 or 21 years old when they are interviewed, an age when many young people 
across the socioeconomic spectrum are still struggling to find their way. This may be particu- 
larly true because the survey was mostly administered during 2009, in the midst of the most 
severe recession in a generation, when the unemployment rate for workers between 16 and 24 
years old was approaching 20 percent — and even higher for young workers with no post- 
secondary education. 
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Appendix A 

Analyses of Survey Response Bias 




Appendix A assesses the reliability of impact results measured by the 21 -month survey. 
It also examines whether the impacts for the survey respondents can be generalized to the im- 
pacts for the research sample. The appendix first describes how the survey sample was selected. 
Then it discusses the response rates for the survey sample and for the program and control 
groups. Next, it examines differences between survey respondents and nonrespondents, fol- 
lowed by a comparison of the two research groups among the survey respondents. 

This appendix concludes — with some caution — that the survey is reliable and that the 
results for the survey respondent sample can be generalized to the research sample. A compari- 
son of the program and control groups among the survey respondents shows few differences in 
background characteristics, and those were largely addressed in the analysis model. Further- 
more, respondents and nonrespondents did not differ in measurable characteristics before ran- 
dom assignment. 



Survey Sample Selection 

The research sample includes 3,074 sample members who were randomly assigned 
from June 2005 through December 2006. Approximately half the full research sample, a total of 
1,507 individuals, was selected to be interviewed for the 21 -month survey. This sample is re- 
ferred to as the “fielded sample” (see Box A.l and Table A.l). With a planned analysis to esti- 
mate a pooled impact where each site is weighed equally, the sample was chosen to minimize 
the variance of estimated impacts in the model. Ideally, this means selecting an equal number of 
program and control group members from each site. However this was not possible with 



Box A.l 

Key Samples 

Research sample. All individuals in the study randomly assigned from June 2005 
through December 2006. 

Fielded Sample. Sample members in the research sample who were selected for the field 
interview. 

Respondent Sample. Sample members who completed the survey. 

Nonrespondent Sample. Sample members in the fielded sample who were not inter- 
viewed because they were not located, refused to be interviewed, or for other reasons. 

Overlap Sample. Sample members who completed both the nine-month and 21 -month 
surveys. 

Nonoverlap Sample. Sample members who completed either the nine-month or 21- 
month survey. 
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Appendix Table A.l 
Fielded Survey Sample 


State 


Total Sample 


Program Group Control Group 


California 


123 




94 


29 


Florida 


157 




95 


62 


Georgia 


188 




94 


94 


Illinois 


190 




95 


95 


Michigan 


189 




95 


94 


Mississippi 


143 




94 


49 


North Carolina 


168 




94 


74 


New Mexico 


92 




66 


26 


Texas 


133 




95 


38 


Wisconsin 


124 




94 


30 


Total 


1,507 




916 


591 



SOURCE: MDRC calculations. 



the wide variation in site size for ChalleNGe. Therefore, a balanced survey sample was chosen 
from each of the larger sites, and all control group members but more program group members 
were chosen in smaller sites. 



Survey Response Rates 

Sample members who were interviewed for the survey are referred to as “survey respon- 
dents,” or the “respondent sample,” while sample members among the fielded sample who were 
not interviewed are known as “nonrespondents,” or the “nonrespondents sample.” A total of 1,196 
sample members, or 79 percent of the fielded sample, completed the survey (80 percent [total = 
736] of program group members and 78 percent [total = 460] of control group members). 

Of the nonrespondent sample, 51 percent (159 out of 311) could not be located for the 
interview. 1 Whenever the response rate is lower than 100 percent, nonresponse bias may occur. 
That is, differences may exist between the respondent sample and the larger, fielded sample, ow- 
ing to differences between the sample members who completed a survey and those who did not. 



'Other respondents were not interviewed because they refused, were in jail, in the military, in rehab, 
or deceased. 
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Comparisons Between Respondents and Nonrespondents 
Within the Survey Sample 

In order to examine whether there are systematic differences between those who re- 
sponded to the survey and those who did not, Appendix Table A.2 presents an analysis of se- 
lected baseline characteristics of survey respondents and nonrespondents. Overall, the two 
groups are similar. However, a few statistically significant differences emerge. Respondents 
were less likely to have been arrested and convicted before random assignment and, consistent 
with decreased involvement with the criminal justice system, were less likely to have the crimi- 
nal justice system suggest application to ChalleNGe. During the survey fielding, the survey firm 
reported difficulty in interviewing sample members who were incarcerated and tried to follow 
up and interview them when released, if possible. For the next survey wave, the survey firm is 
conducting interviews with those who are currently incarcerated, so this difference will likely 
shrink or disappear. The respondent group was also more likely to give education and training 
as a reason for applying to ChalleNGe. Conversely, prior to random assignment a significantly 
higher percentage of the respondent group were considered obese, based on their reported 
height and weight. 



Comparison Between the Research Groups in the Survey 
Respondent Sample 

Although random assignment research designs minimize the potential bias, there is the 
possibility that the characteristics of the research groups will differ due to the selective nature of 
the survey response process. If this is true, the reliability of impact estimates for the respondent 
sample may be affected. 

Appendix Table A.3 shows selected baseline characteristics of the program and control 
group members who responded to the 21 -month survey. Differences emerge among respon- 
dents for highest grade completed and prior arrest/conviction. For highest grade completed, the 
characteristic was included in the analysis model to control for the variation between the two 
groups. For prior arrest/conviction, the program group fared worse, so would likely cause the 
effects of the program to be underestimated, if anything. A logistic regression predicting treat- 
ment status among respondents confirmed that the control and program groups are similar; 
baseline characteristics cannot predict treatment status (the F-statistic testing joint significance 
was not significant). 



Comparison Between the Survey Waves 

Lastly, it is inevitable to want to compare the results of this survey with those of the ear- 
lier survey wave to understand the potential influence of the ChalleNGe program over time. In 
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Appendix Table A.2 

Selected Baseline Characteristics of Survey Respondents and Nonrespondents 



Characteristic (%) 


Respondent 

Group 


Nonrespondent 

Group 


Total 


Male 


87.9 


93.0 


89.0 


Average age (years) 


16.7 


16.7 


16.7 


Race/ethnicity 11 


Hispanic 


18.6 


17.1 


18.3 


White 


43.0 


44.0 


43.2 


Black 


32.1 


35.6 


32.8 


Other 


0.5 


0.2 


0.5 


Highest grade completed 


8th grade or lower 


14.6 


14.3 


14.5 


9th grade 


31.6 


31.1 


31.5 


10th grade 


36.0 


38.8 


36.6 


1 1th grade 


17.1 


15.4 


16.8 


12th grade 


0.7 


0.5 


0.7 


Usual grades received in school 


Mostly As and Bs 


3.9 


4.7 


4.1 


Mostly Bs and Cs 


17.9 


20.8 


18.5 


Mostly Cs and Ds 


41.6 


36.4 


40.5 


Mostly Ds and Fs 


46.8 


48.2 


47.1 


Has/had Individual Education Plan (IEP) 


32.5 


28.7 


31.7 


Ever suspended from school 


81.8 


82.7 


81.9 


Ever arrested 


31.5 


40.7 


33.4 *** 


Ever convicted 


17.5 


24.5 


9 *** 



Who first suggested you should apply for ChalleNGe? (%) 



Yourself 


24.3 


24.8 


24.4 


A relative 


46.9 


48.7 


47.2 


A school official 


17.5 


14.3 


16.9 


The justice system 


7.0 


12.0 


g o *** 


Reasons for applying to ChalleNGe? 


Want a high school diploma/GED certificate 


81.9 


77.3 


81.0 * 


Want to go to college/get more training 


43.5 


36.1 


42.0 ** 


Want to get a job 


39.4 


36.1 


38.7 


Want to join the military 


33.7 


31.2 


33.2 


Want to get life on track 


79.4 


80.8 


79.7 


Overall health very good or excellent 


67.2 


69.1 


67.6 


Taking any medication 


23.5 


21.1 


23.0 


Overweight (BMI 25-29) b 


20.6 


23.0 


21.1 



(continued) 
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Appendix Table A.2 (continued) 



Characteristic (%) 


Respondent 

Group 


Nonrespondent 

Group 


Total 


Obese (BMI 30+) b 


11.5 


5.7 


10.3 *** 


Ever drink alcohol or use drugs 


38.4 


36.9 


38.1 


Sample size 


1,196 


311 


1,507 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 



NOTES: Missing values are not included in individual variable distributions. 

Rounding may cause slight discrepancies in sums and differences. 

Distributions may not add to 100 percent where categories are not mutually exclusive. 

a Race/ethnicity categories were constructed by counting as Hispanic those who checked both Hispanic 
and black or white. None of these sample members are counted as multiracial and grouped in the "other" 
category. 

b Body Mass Index (BMI) is a measure of body fat based on height and weight that applies to both adult 
men and women. BMI is calculated by dividing a person's weight by his or her height squared. A person is 
defined as overweight if his or her BMI is between 25 and 29.9. A person is defined as obese if his or her 
BMI is 30 or higher. 



order to compare the results of these two surveys, when possible, it is important to understand 
whether reductions in impacts over time are owing simply to who responded to the survey. 
More than half of the 21 -month survey respondent sample (55 percent) also completed the 
shorter nine-month survey. These individuals are referred to as the “overlap sample.” Those 
individuals who completed only one of the two surveys are referred to as the “nonoverlap sam- 
ple.” 



Appendix Table A.4 looks at the impacts for key ChalleNGe outcomes at the two time 
points. By comparing the change in impacts for the overlap and nonoverlap samples, one can 
understand if one of the groups is driving the change in results over time. As the left panel of 
Table A.4 shows, the change in impacts, using weights, from the first to the second survey is 
not due to one particular sample. Thus, the differences are due to something beyond the chang- 
ing survey sample. In addition, since the weighting scheme differed in the analysis of each sur- 
vey wave, the right panel of Table A.4 shows the unweighted impacts with a similar variation in 
the differences over time for the two samples. 
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Appendix Table A.3 

Selected Baseline Characteristics of Program and Control Group Members, 
among Survey Respondents 

Program Control 

Characteristic (%) Group Group Total 



Male 


87.7 


88.2 


87.9 


Average age (years) 


16.7 


16.8 


16.7 


Race/ethnicity J 








Hispanic 


19.9 


16.8 


18.7 


White 


43.0 


43.1 


43.0 


Black 


30.5 


34.5 


32.1 


Other 


0.3 


0.9 


0.5 


Highest grade completed 






*** 


8th grade or lower 


12.4 


17.7 


14.6 


9th grade 


31.7 


31.3 


31.6 


10 th grade 


35.5 


36.8 


36.0 


1 1th grade 


19.4 


13.8 


17.1 


1 2th grade 


0.9 


0.4 


0.7 


Usual grades received in school 








Mostly As and Bs 


3.7 


4.1 


3.9 


Mostly Bs and Cs 


18.1 


17.7 


17.9 


Mostly Cs and Ds 


42.9 


39.7 


41.7 


Mostly Ds and Fs 


46.7 


46.9 


46.8 


Has/had Individual Education Plan (IEP) 


31.7 


33.8 


32.5 


Ever suspended from school 


82.2 


81.1 


81.8 


Ever arrested 


34.1 


27.7 


31.7 ** 


Ever convicted 


19.3 


14.9 


17.6 * 


Who first suggested you should apply for ChalleNGe? (%) 








Yourself 


25.4 


22.6 


24.3 


A relative 


45.6 


48.8 


46.8 


A school official 


17.3 


17.9 


17.5 


The justice system 


6.8 


7.4 


7.0 


Reasons for applying to ChalleNGe? 








Want a high school diploma/GED certificate 


82.5 


81.1 


81.9 


Want to go to college/get more training 


43.2 


44.0 


43.5 


Want to get a job 


39.1 


39.8 


39.4 


Want to join the military 


33.2 


34.5 


33.7 


Want to get life on track 


78.0 


81.5 


79.3 


Overall health very good or excellent 


68.1 


65.9 


67.2 


Taking any medication 


24.6 


21.7 


23.5 


Overweight (BMI 25-29) b 


21.3 


19.7 


20.6 


(continued) 



48 






Appendix Table A.3 (continued) 



Characteristic (%) 


Program 

Group 


Control 

Group 


Total 


Obese (BMI 30+ ) b 


11.6 


11.3 


11.5 


Ever drink alcohol or use drugs 


40.1 


35.7 


38.4 


Sample size 


736 


460 


1,196 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 



NOTES: Missing values are not included in individual variable distributions. 

Rounding may cause slight discrepancies in sums and differences. 

Distributions may not add to 100 percent where categories are not mutually exclusive. 

“Race/ethnicity categories were constructed by counting as Hispanic those who checked both Hispanic 
and black or white. None of these sample members are counted as multiracial and grouped in the "other" 
category. 

b Body Mass Index (BMI) is a measure of body fat based on height and weight that applies to both adult 
men and women. BMI is calculated by dividing a person's weight by his or her height squared. A person is 
defined as overweight if his or her BMI is between 25 and 29.9. A person is defined as obese if his or her 
BMI is 30 or higher. 
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Appendix Table A.4 



Selected Impacts among Survey Respondents in Wave 1 and Wave 2 Surveys, by Weights 



Outcome 


W1 Impact 


Weighted 

W2 Impact 


Difference 


W1 Impact 


Not Weighted 

W2 Impact 


Difference 


Received high school diploma/GED certificate (%) 


Overlap 


35.3 


24.7 


10.6 


35.6 


24.3 


11.3 


Nonoverlap 


37.9 


25.7 


12.2 


37.7 


22.8 


14.9 


Currently taking college courses (%) 


Overlap 


8.4 


3.8 


4.6 


8.5 


4.3 


4.2 


Nonoverlap 


8.9 


6.2 


2.7 


8.8 


6.4 


2.4 


Working full time (%) 


Overlap 


9.5 


7.5 


2.0 


9.0 


5.4 


3.7 


Nonoverlap 


10.5 


-3.9 


14.4 


10.9 


-1.4 


12.3 


Arrested (%) 


Overlap 


-6.0 


-3.3 


-2.6 


-6.1 


-2.5 


-3.6 


Nonoverlap 


-8.7 


-1.9 


-6.8 


-8.3 


1.9 


-10.2 


Convicted (%) 


Overlap 


-4.1 


-6.4 


2.2 


-4.3 


-4.7 


0.4 


Nonoverlap 


-6.2 


-7.5 


1.3 


-6.2 


-1.9 


-4.3 


Body Mass Index* 1 


Overlap 


-0.4 


0.4 


-0.8 


-0.4 


0.1 


-0.5 


Nonoverlap 


0.9 


0.5 


0.4 


0.9 


0.2 


0.6 


Living in own home (%) 


Overlap 


-0.6 


-0.4 


-0.2 


-0.8 


-0.1 


-0.6 


Nonoverlap 


-1.6 


4.1 


-5.7 


-1.3 


7.2 


-8.5 


Self-reported health good or excellent (%) 


Overlap 


10.7 


1.9 


8.8 


10.3 


5.1 


5.3 


Nonoverlap 


1.0 


-3.0 


4.0 


1.6 


-1.4 


3.0 



(continued) 





Appendix Table A.4 (continued) 



SOURCE: MDRC calculations from responses to the 9-month and 21 -month surveys. 

NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member characteristics. Statistical 
significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Rounding may cause slight discrepancies in sums and differences. 

a Body Mass Index (BMI) is a measure of body fat based on height and weight that applies to both adult men and women. BMI is 
calculated by dividing a person's weight by his or her height squared. A person is defined as overweight if his or her BMI is between 25 
and 29.9. A person is defined as obese is his or her BMI is 30 or higher. 
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National Guard Youth ChalleNGe Program 
Appendix Table B.l 

Supplemental Impacts on Delinquency and Criminal Activity in the Last 12 Months 



Outcome (%) 


Program 

Group 


Control 

Group 


Impact 


P- Value 


Self-reported arrests/charges 










Arrested 


26.3 


25.4 


0.9 


0.719 


Arrested only once 


17.4 


15.2 


2.3 


0.309 


Arrested more than once 


8.9 


10.2 


-1.3 


0.450 


Charged with any crime 


25.0 


24.7 


0.3 


0.917 


Charged with a violent crime 


3.4 


3.6 


-0.2 


0.842 


Charged with a property crime 


4.1 


5.4 


-1.3 


0.316 


Charged with a drug crime 


2.9 


5.3 


-2.5 ** 


0.030 


Charged with a public order crime 


17.5 


13.0 


4.6 ** 


0.034 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21-month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 
10 percent. 

Rounding may cause slight discrepancies in sums and differences. 
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Table B.2 

Types of Self-Reported Delinquency 



Outcome (%) 


Never 


Three or 
Once Twice four times 


Five or 
more times 


Violent incidents 


Got into a serious physical fight 


51.4 


15.6 


12.4 


10.3 


10.4 


Got into a fight with another group 


78.7 


10.0 


5.3 


3.3 


2.8 


Hurt someone badly enough to need medical attention 


74.2 


14.2 


6.7 


2.6 


2.4 


Used or threatened to use a weapon to get something 


96.7 


1.8 


0.8 


0.3 


0.3 


Property incidents 


Deliberately damaged property 


87.0 


7.1 


3.6 


1.6 


0.8 


Stole something worth more than $50 


93.7 


3.3 


1.3 


1.0 


0.7 


Stole something worth less than $50 


89.7 


5.1 


2.3 


1.5 


1.4 


Entered a house without owner's permission in order 
to steal something 


96.0 


3.0 


0.8 


NA 


0.3 


Drove a car without owner's permission 


90.6 


4.3 


2.6 


0.9 


1.6 


Bought, sold, or held stolen property 


92.2 


4.0 


2.1 


0.7 


1.1 


Used someone's credit card or bank card without their permission 


98.2 


0.9 


0.4 


0.2 


0.3 


Deliberately wrote a bad check 


97.8 


1.3 


0.3 


0.3 


0.3 


Other incidents 


Sold marijuana or other drugs 


87.5 


2.6 


2.1 


1.3 


6.5 


Sample size (total = 1,196) 













SOURCE: MDRC calculations from responses to the 21 -month survey. Delinquency questions from Haynie 

( 2001 ). 

NOTES: Rounding may cause slight discrepancies in sums and differences. 
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Appendix C 

Supplementary Tables Analyses on Life-Coping, 
Leadership, and Group Skills 




As discussed earlier, modest but promising significant differences were found be- 
tween program and control groups on a cluster of single items assessing life-coping, leader- 
ship, and group skills. For the purpose of this report, only single survey questions were dis- 
cussed so that they could be evaluated alongside comparable items from other national data 
surveys (See Table C.l). 

In addition to these single item comparisons, preliminary exploratory analyses were 
conducted using composite measures created from factor analysis (See Table C.2). The individ- 
ual survey items included in this phase of evaluation were selected from existing measures, as 
well as pilot measures under development by the MacArthur Research Network on Transitions 
to Adulthood. To create composite measures, factor analyses were performed to test the rela- 
tionship between the observed survey item responses and their hypothesized underlying latent 
constructs. Exploratory factor analysis produced a total of 12 composite measures. Table C.2 
lists the items by composite measure; the factor loadings were based on a varimax rotation (an 
orthogonal rotation method), and a priori theory. 

Impacts of these composite measures, shown in Table C.3, are regression-adjusted us- 
ing ordinary least squares and control for select baseline respondent characteristics. Results are 
summarized below: 

• Statistically significant program impact on leadership and problem-solving 
s ki lls and positive outlook and perseverance but not for sense of control or 
time-management abilities 

• Statistically significant program impact found for emotional regulation and 
impulsivity but not for thrill-seeking behaviors 

• Statistically significant program impact found on interdependence and 
group interactions but not for diverse relationships, tolerance, or ra- 
cial/ethnic sensitivity 

In sum, in these exploratory analyses using composite measures, ChalleNGe partici- 
pants report more skills related to taking responsibility for and control of themselves and their 
impulses, solving problems and setting goals, and being productive, harmonious group mem- 
bers. Although we cannot be certain at this point, ongoing meaningful, youth-relevant conversa- 
tions between the participants and their mentors, as well as other program staff and peers may 
be an important mechanism through which such benefits accrue. This includes contact with 
counselors, who play a special role in helping participants build their life skills, including teach- 
ing modules on such topics as anger management, career planning, coping and problem-solving 
skills, and job search skills. 
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Appendix Table C.l 

Comparison Statistics for Leadership and Life-Coping Survey Items 

ChalleNGe Mean Comparison Mean 

Item (SD) (SD) 

If you were to compare yourself now with the way you were at the time of 
random assignment, how well would each of the following statements 
describe you ? 3 



I have learned about setting priorities. 


3.5 


2.1 




(.8) 


(-1.1) 


I have learned about organizing time and not putting things off. 


3.5 


2.3 




(.8) 


(-1.1) 


I have observed how others have solved problems and learned from them. 


3.6 


2.2 




(■7) 


(-1.1) 


I have learned about the challenges of being a leader. 


3.6 


2.2 




(■7) 


(-1.1) 


I have had the experience of others counting on me. 


3.6 


2.1 




(■7) 


(-1.1) 


I have learned how to control my temper better. 


3.5 


2.4 




(.8) 


(-1.2) 


I have become better at dealing with my fears and worries. 


3.6 


2.5 




(■7) 


(-1.2) 


I have tried new ways of acting around people. 


3.2 


2.5 




(1.0) 


(-1.1) 


The next set of questions asks how you do things in group situations. 
For each of the following, please answer whether in group situations 
you do this never, rarely, sometimes, often or always 15 


You try to present your ideas without criticizing the ideas of others. 


3.4 


4.1 




(1.2) 


(-0.8) 


You encourage different points of view without worrying about agreement. 


3.6 


3.9 




(1.2) 


(-0.9) 


You try to consider all points of view or possible options before 


forming an opinion or making a decision. 


4.0 


4.1 




(1.0) 


(-0.8) 



SOURCE: Hansen and Larson, 2005. 



NOTES: “Comparative means from a sample of 1,822 youth, part of instrument development sample for the 
Youth Experience Survey. These youth included llth-graders from 19 diverse high schools and 
approximately matched the population of Illinois in ethnicity, socioeconomic status, and urban vs. suburban 
vs. rural residence. Item response set: Not at all (1), A little (2), Quite a bit (3), and Yes, definitely (4). 

•“Comparative means from a sample of 2,279 youths, ages 16-30, part of the Americorps evaluation. 
Americorps item response set adapted for ChalleNGe: Never (1), Not very often (2), Sometimes (3), Very 
often (4), and Always (5). 
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National Guard Youth ChalleNGe Evaluation 
Appendix Table C.2 

Items and Factor Loadings for Leadership and Life-Coping Scales 





Time 


Emotional 




Life-Coping Skills 


Management 


Regulation 


Sense of Control 


Learned to set priorities 


0.318 






Learned to organize time and not put things off 
Learned to better control temper 


0.732 


0.379 




Has become better at dealing with fears and worries 




0.419 




Has become more open to doing new things 




0.412 




Has tried new ways of acting around people 




0.511 




Has started thinking about who he/she wants to be 
Believes other people determine what he/she can do 




0.381 


0.500 


Believes he/she has little control over what happens 






0.645 


Believes he/she has no way of solving problems 






0.524 


Cronbach alpha coefficient 


0.56 


0.63 


0.53 





Positive 

Outlook 


Impulsivity 


Thrill-Seeking 


Believes people think he/she has useful talents 


0.361 






Can find something positive in even the worst situation 


0.389 






Rarely gives up when things get tough 
Usually acts from "gut feeling" and doesn't think 


0.366 






much about the consequences of actions 




0.463 




Deals with problems in his/her own way 




0.357 




Lives life without much thought for the future 




0.413 




Likes doing things without strict rules 




0.402 




Changes interests a lot because his/her attention shifts 
Tries new things just for fun or excitement 
When nothing is happening, usually starts 




0.442 


0.397 


looking for excitement 






0.869 


Cronbach alpha coefficient 


0.42 


0.56 


0.59 



(continued) 
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Appendix Table C.2 (continued) 



Leadership and Group Skills 


Leadership 


Interdependence 


Group Relations 


Has observed how others have solved problems 








and learned from them 


0.414 






Has learned how to solve problems in different ways 


0.691 






Has learned the challenges of being a leader 


0.475 






Has had the experience of others counting 








on him/her 


0.370 






Has someone to ask for help when he/she needs it 




0.492 




Is able to ask for help when he/she needs it 




0.641 




People know they can count on him/her for help 




0.492 




Has people that he/she can count on 




0.722 




Feels that he/she is important to others 




0.576 




Has people in his/her life who matter to him/her 




0.383 




People seek his/her advice and support 




0.452 




Seeks advice and support from others 




0.524 




Tries to present his/her ideas without 








criticizing others 






0.641 


Encourages different points of view without 








worrying about agreement 






0.649 


Tries to consider all points of view or possible 








options before forming an opinion or making 








a decision 






0.500 


Cronbach alpha coefficient 


0.63 


0.75 


0.62 





Intergroup 


Racial 




Relationships 


Senstivity/Prejudice 


Often socializes with members of other racial or ethnic groups 


0.385 




Feels comfortable in groups of people different than him/her 
Thinks before doing or saying something prejudiced 


0.941 


0.581 


Recognizes when he/she does things that are prejudiced 
Stops others when they say negative things about 




0.614 


other racial or ethnic groups 




0.227 


Cronbach alpha coefficient 


0.56 


0.48 



SOURCE: MDRC calculations from responses to the 21 -month survey. 

NOTES: Only factor items with loadings greater than or equal to 1.301 are shown, based on factor analysis. 
Factor loadings indicate items that were used to create the respective scales. Items were included on the 
factors on which they most highly loaded. 

See Table C.3 for impact table using the scales created here. 
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National Guard Youth ChalleNGe Program 
Table C.3 



Additional Exploratory Impacts on Life-Coping, Leadership, and Group Skills 



Outcome 


Program 

Group 


Control 

Group 


Impact 


P- Value 2 


Life-coping skills 


Time management 


7.1 


7.0 


0.1 


0.106 


Sense of control 


12.8 


12.6 


0.3 


0.115 


Positive outlook/perseverance 


22.7 


22.1 


0.5 *** 


0.001 


Emotional regulation 


17.9 


17.5 


Q 4 *** 


0.007 


Impulsivity 


12.1 


13.1 


Q *** 


0.000 


Thrill-seeking 


6.8 


6.9 


-0.2 


0.211 


Leadership and group skills 


Leadership and problem solving 


14.7 


14.4 


Q 3 *** 


0.009 


Interdependence 


37.5 


36.9 


Q ^ *** 


0.006 


Constructive group interactions 


11.2 


10.8 


0.3 ** 


0.043 


Cultivates diverse relationships 


8.7 


8.7 


0.0 


0.948 


Racial/ethnic senstivity 


13.0 


12.9 


0.1 


0.422 


Tolerance 


3.8 


3.8 


0.0 


0.151 


Sample size (total = 1,196) 


736 


460 







SOURCE: MDRC calculations from responses to the 21 -month survey. 



NOTES: Estimates are regression-adjusted using ordinary least squares, controlling for sample member 
characteristics. Significance levels are indicated as follows: *** = 1 percent, ** = 5 percent, and * = 10 
percent. 

Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the 
standard errors for all impacts with a p-value of 0.000 (presented in the order in which they appear in the table): 
.0233 



No statistically significant differences were found between program and control groups 
in items evaluating youths’ sense of control or time-management abilities. The sense of control 
items assessed how much youth felt in charge of their behaviors and choices (for example, 
“Other people determine most of what I can and cannot do”) as well as events (for example, “I 
have little control over the things that happen to me”). Time-management items focused on set- 
ting priorities and procrastination (for example, “I have learned about organizing time and not 
putting things off’). Even though there was no evidence of statistically significant differences, 
trends were in predictable directions, with program youth reporting higher levels of self-control 
and time -management abilities. These findings may be related to the organizational structure 
and military nature of the ChalleNGe program. In addition, no differences were found in the 
thrill-seeking measure, which evaluated the tendency to seek out new activities for fun or ex- 
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citement and may be more developmental^ driven. That is, most youth this age have the ten- 
dency to seek out new exciting adventures and opportunities, and an intervention program like 
ChalleNGe might not necessarily affect that. 

Lastly, there was no evidence of statistically significant differences between program 
and control groups on intergroup relation measures, specifically diverse relationships, toler- 
ance, or racial/ethnic sensitivity. However, any differences in means, although not statistically 
significant, were in predictable directions. Since only some programs have a heterogeneous mix 
of cadets, the lack of program impact may be because there was lack of opportunity in more 
homogeneous sites. Site -by-site comparisons might be useful in future analyses (looking at pro- 
gram effects within more diverse sites versus more homogenous sites where opportunities are 
fewer to cultivate this capacity) for this cluster of measures. 

Future research should assess the underlying pathways through which ChalleNGe con- 
tributes to some of these changes, as well as further substantiate the impact of the program (and 
different components of the program) on these types of psychosocial maturity skills. Future fol- 
low-up is needed to fine-tune analysis of the pilot aggregate measures as well as to determine 
the longer-term program effects on these coping skills and the relationship of these skills to oth- 
er key outcomes for participants in the program, such as work and academic success. 
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Appendix D 

Items and Factor Loadings for Civic Engagement Scales 




National Guard Youth ChalleNGe Evaluation 



Appendix Table D.l 



Items and Factor Loadings for Civic Engagement Scales 



Social Responsibility 


Good Citizen 


Civil Justice 


It's important to assist those who are most in need of help 
Everyone should be concerned with 


0.446 




national, state, and local issues 


0.723 




It's important to challenge inequalities 
It's important for people to work with 




0.439 


local organizations and government to solve problems 
It's important to protest when something 




0.479 


in society needs changing 
It's important to follow rules and laws 




0.526 


Cronbach alpha coefficient 


0.49 


0.45 



Civic Commitment 


Justice-Oriented 

Citizen 


Conventional 

Citizen 


It's important to respondent to help those who are less fortunate 
It's important to respondent to help improve relations 


0.451 




between racial and ethnic groups 


0.872 




It's important to respondent to serve his/her country 




0.409 


It's important to respondent to practice his/her religion 




0.408 


It's important to respondent to be a leader in his/her community 




0.599 


It's important to respondent to help solve environmental problems 




0.593 


It's important to respondent to get a job to contribute to society 
It's important to respondent to respect people, no matter who they are 
It's important to respondent to stick up for his/her rights 
if pushed around 




0.681 


Cronbach alpha coefficient 


0.69 


0.65 



SOURCE: MDRC calculations from responses to the 21 -month survey. 

NOTES: Only factor items with loadings greater than or equal to 1.301 are shown, based on factor analysis. 
Factor loadings indicate items that were used to create the respective scales. Items were included on the factors 
on which they most highly loaded. 



67 











Appendix E 

Supplementary Subgroup Tables 
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Has high school diploma or GED certificate and is currently 

involved in any of the above activities 1 (%) 46.0 27.5 18.5 *** 0.000 43.9 15.0 28.9 *** 0.000 
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indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 
Rounding may cause slight discrepancies in sums and differences. 
a Conviction categories from Langan and Levin (2002). 
b See full list of types of delinquency in Box 2. 
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“Standard errors are presented in this report for all impacts with a p-value of 0.000. Following are the standard errors for all impacts with a p-value of 0.000 
(presented in the order in which they appear on the table): 4.854. 

b "Other living arrangement" includes: living in a group home or homeless shelter, not living anywhere (homeless), or any other arrangement not specifically 
listed. 
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Appendix Table E.7 (continued) 
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Rounding may cause slight discrepancies in sums and differences. 
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Has high school diploma or GED certificate and is currently 

involved in any of the above activities (%) c 41.7 12.3 29.4 *** 0.000 47.6 28.0 19.6 *** 0.000 




Delinquency Level 

Ever Arrested/Convicted Never Arrested/Convicted 
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indicated as follows: *** = 1 percent; ** = 5 percent; *=10 percent. 
Rounding may cause slight discrepancies in sums and differences. 
“Conviction categories from Langan and Levin (2002). 
b See full list of types of delinquency in Box 2. 
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Appendix Table E.14 (continued) 
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National Guard Youth ChalleNGe Program 
Appendix Table E.15 
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Rounding may cause slight discrepancies in sums and differences. 

“"Other living arrangement" includes: living in a group home or homeless shelter, not living anywhere (homeless), or any other arrangement not 
specifically listed. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the 
active communication of its findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successful Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




